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ABSTRACT 


A broad look at the problems of convolutional encoding 
and sequential decoding of digital data for reliable trans- 
mission over a noisy channel is presented. Various dis- 
tance measures for convolutional encoders are carefully de- 
fined. A distance measure called the free distance, and 
introduced by Massey [9] , McEliece and Rumsey [10] , and 
Neumann [ll] , is shown to be a more important parameter 
for codes used with sequential decoding than the conven- 
tional minimum distance. 

The notion of equivalent encoders is carefully con- 
sidered, and a new definition of encoder equivalence which 
preserves the distance properties of the encoder is given. 
For those non-systematic encoders that do not have a system- 
atic equivalent which preserves distance properties, a 
method is presented for converting the encoder to a better 
systematic encoder. Also, general parity-check matrices 
and syndrome forming circuits for non-systematic encoders of 
all rates are presented. 

A new lower bound on free distance is given, and 
McEliece and Rumsey' s [10] upper bound is generalized. Also, 
a new lower bound on definite decoding minimum distance, for 
both systematic and non-systematic codes, is derived, as 
well as Gilbert lower bounds on feedback decoding minimum 
distance for two simply implemented subclasses of convolu- 
tional codes. Finally, some new methods for calculating the 
free distance are discussed. 

Construction algorithms are given which produce longer 
good codes than any previously known. A sequential de- 
coding system was simulated for both the Gaussian and binary 
symmetric channels. Various codes were tested over differ- 
ent channels for their performance as regards decoding prob- 
ability of error and decoding speed (number of computations) . 
A simply implemented non-systematic rate 1/2 code was found 
to be far superior to all other codes tested with the same 
constraint length. 

Finally, a technique introduced by McEliece [lo] for 
constructing good convolutional codes from known block codes 
was extended, and some convolutional codes discovered by 
Wyner [34] and Sullivan [35] were discussed. 
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I . Introduction 


A. Background 

Convolutional codes were first introduced by Elias in 
1955 [l] . Recently, Viterbi [ 2 ] has shown that a lower 
probability of decoding error is achievable with convolu- 
tional codes than with block codes of the same rate and com- 
parable complexity used over a memoryless channel. Another 
advantage of convolutional codes over block codes is the ex- 
istence for the former of a general, simply instrumented, 
and near-optimal decoding algorithm, called sequential de - 
coding , whose evolution began with Wozencraft in 1957 [3] . 
Sequential decoding is probabalistic in nature, and the same 
basic sequential decoding algorithm is applicable to a very 
general class of codes, called tree codes , of which convo- 
lutional codes are a subclass. At the present time, sequen- 
tial decoding of convolutional codes represents the best 
practical means of employing long codes on the additive 
white Gaussian noise channel and on many other memoryless 
channels. Algebraic decoding techniques for convolutional 
codes, such as Massey's threshold decoding [4] , also pro- 
vide good coding systems for certain channels with memory, 
and for short codes on memoryless channels. 

B. Review of Convolutional Encoding 

It is convenient to define a convolutional code after 
first making precise what is meant by a convolutional en- 
coder. A more general type of encoder is defined first. 
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Definition 1.1 An (N,K) general convolutional encoder G 
is an invertible K-input, N-output realizable linear finite- 
state machine (FSM) which is in the zero state in the infin- 
ite past. | 

Figure 1.1 shows an (N,K) general convolutional en- 
coder. At each unit of time the encoder transforms the 
block of K input digits into a block of N output digits. 
Realizable means that the encoder output cannot depend upon 
future inputs. Invertibilitv [5] means that the input can 
be recovered, perhaps with delay, from the encoder output 
and is the elementary condition for an encoder to be useful. 
Without an inverse, decoding could be ambiguous, even in 
the absence of noise. If G is time -invariant, it is called 
a fixed convolutional encoder , or simply a convolutional 
encoder . When the encoder G is time- invariant, definition 
1.1 reduces to Forney's [6] definition of a convolutional 
encoder. 

Consider a semi-infinite (or one-sided) sequence 

a , a , . . . of digits over a finite field GF ( q) , where 
n n+I 

- °° < n < + °o. The transform A(D) of this sequence is de- 
fined to be the formal power series 

A(D) = a n D n + a n+1 D n+1 + ... . 

If a . =0 for all i < 0, then the sequence is called causal, 

l 

and if the sequence has only a finite number of non-zero 
digits, it is called finite . A polynomial in the indeter- 
minate D over a finite field GF(q) is thus seen to be iden- 
tified as the transform of a causal, finite sequence. The 
degree of a polynomial is the highest power of D in the 
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polynomial, and the delay of a polynomial is the highest 
power of D which it contains as a factor. By convention, 
the polynomial 0 has degree - °°and delay +°°. 

P(D) 

A rational function is a ratio of two polynomials - , 

where Q(D) / 0 and the division is assumed to begin with the 
lowest degree terms of each polynomial. A sequence is called 
rational if its transform is a rational function. The ra- 
tional sequence with transform j is causal if and only if 
n' < n, where n‘ is the delay of Q(D) and n is the delay of 
P (D) . Causal rational sequences are called realizable since 
they can be produced by an autonomous linear finite state 
machine. A realizable function is the transform of a causal 
rational sequence. 

The K input sequences to any encoder will be restricted 


to be rational sequences. The set of K input sequences will 
be represented by the semi -inf inite row vector x = fx , x^ + ^ , 

"1 f /TV / V / 1 ( /*.% L , _ . 


■■■] = [ x n (1> ”- 


x i " [ X i • • • x i'^ ; J row vector or K -tuple of input 

digits at time unit i, and ^ is the input digit at time 
tin 

unit i in the j input sequence, n<i<°°,i^j<K. is 


... j , where n is the least integer such that x n / 0, 


* * ’ x i 


called the information sequence since it represents the data 


to be encoded. The transform of the information sequence 


will be written as the K-tuple x(°) = jj 
x (K) (D) j , where x^ (D) = x n ^D n + x. 


x (1) (D) , x (2) (D) , . 

(j) n+1 (j! 

n+1 D + x n+2 
. . .th ^ 


+ ..., 1 < j < iK, is the transform of the j input 


sequence. 
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•] 

. . . ] / where = \y ± 


Since all encoders are realizable and the input se- 
quences are rational, the N output sequences of any encoder 
are also rational and may be represented by the semi-infin- 
ite row vector £ =[v S^j, ••-] =[y n (1> . . .y n W 

v (i) „ (n> (i) (n> i r (i) 

Y n+1 ••* Y n+1 Y n+2 ••• Y n+2 { ••• J / where y.. =\y^ 

(N) 1 

. • • y i is the row vector or N-tuple of output digits at 

time unit i, and y^-^ is the output digit at time unit i in 
tin 

the j output sequence, n < i < oo , x < j < N. y is called 
the transmitted sequence or codeword since it represents the 
data which is actually to be transmitted over the channel. 
The transform of the transmitted sequence will be written as 


the N-tuple 


Y(D) =[; 


y (1): (D),y (2) (D) , ..., y (N * (D) 


] - 


where 


(d) 


y ( J *D n + y 1 
■'ll ■'n+l 


(J, ° n+1 + y n+2 


(j) D n+2 + i < j < 


N, is the transform of the j output sequence. 

The encoding equations for a general convolutional en- 
coder over the set of all rational input sequences can be 
written as 


= I % (i - t) • (1) 

1=0 

where G^(u) is a KxN matrix of elements from GF(q> , 0 < < °° , 

n < u <°o , and all operations are performed over GF ( q) . 

In the remainder of this thesis only causal rational 
sequences will be allowed as inputs, except when it is ex- 
plicitly stated that all rational sequences are to be con- 
sidered as inputs. In this case n £ 0 and the information 
and transmitted sequences will be assumed to begin at time 
unit 0, i.e., x . . .J =^x Q ^ . . .x Q ^ x^ 1 *... 
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Cl (K) ...]and Y = [zo-Zj-Zj.... ] = [y 0 (1) • . .y 0 (N, yi a) . . 
^ ^ y 2 ^ • • *Y 2 ^ * * ' ] * Hence the encoding equations : 


Hence the encoding equations for 


a general convolutional encoder over the set of all causal 
rational input sequences can be written as 


Z = G / 


where 


G^O) G^O) G 2 (0) ... 

0 Gq (1) G^l) G 2 (l) ... 


0 Gq ( 2) G x (2) G 2 (2) .. 


is called the generator matrix and 


g^i (1) (u) (u) ... ( u > 

(1) , . (2) . v (M) 

g^ 2 (u) g^ 2 (u) ... g^ 2 (u) 

G^(u) = . . . ( 4 j 

• • • 

• • • 

rr (1) / 11 v ~ (2) / i (N) 11 

is a K x N matrix of elements from GP(q) , 0 <; u, <L < °o . 
Definition 1.1 requires that each sequence g ^ ( u ) , ^ 

ggi (u) / . . . / l^i^K/l^j^N/O^ u<oo , be realizable. 
These sequences are called generator sequences . 

The generator matrix for a fixed convolutional encoder 
is written as 


— 0 "I “2 **• 

- -0-1-2 **• 

0 0 G 0 % G 2 . . 
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where 


% - 


g tl (1> ^1 (2> 
g t2 U) ^2 (2> 


* %1 

* %2 


(N) 

(N) 


, 0 < l < oo , 


( 6 ) 


_ (1) (2> (N) 

%K ’ * * g -iK J 

and each generator sequence g n . ^ , g, . ^ / g (j) 

ux xx 2 ± 7 • • • / 

1 < i <, K, 1 < j < N, is realizable. It is often convenient 
to represent a fixed convolutional encoder by a K x N matrix 
G (D) , where 

G 1 ^ (D) G 1 * 2 * (D) ... G 1 (D) 


G(D) = 


g 2 (1) (d) g 2 (2) (d) ... g 2 (n * (d) 


G t , (1) (D) G v ( 2) (D) ... G v (N) (D) J 




K 


K 


(7) 


and 


°i (J> »> = 30i 


+ g,.,^ D + g 2± /^ D 2 + 


*li 


is the transform of the generator sequence g 


2i 


(j) 


01 

(j) 


(j) 


J li 


( 8 ) 

(j) 


, ...,l£i<K, l<j£N. Each G^- 1 ' (D) is called 


a generator function . Then the encoding equations can be 
written in D-operator form as 

y(D) = x(D) £(D) , (9) 

where all operations are performed over GF(q). 
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The following definitions are due to Forney [6] and 
apply to both time-varying and fixed convolutional codes. 
Definition 1.2 The convolutional code C generated by a 
convolutional encoder G is the set of all output sequences 
y of G produced by the set of all rational input sequences 
x to G. | 

Definition 1.3 Two encoders are said to be equivalent if 
they generate the same convolutional code. | 

The rate R of a general convolutional encoder is de- 
fined to be — . The time-u memory order m(u) of a general 
convolutional encoder is defined as 


m(u) 


max { i | G. (u) ^ 0 } 

0<i<°° 


( 10 ) 


I « if no such i exists. 

The memory order m of the encoder is then defined as 


max | m(u) } if m(u) is finite for all u 
0<u <oo ( 11 ) 


| o° otherwise. 

For fixed convolutional encoders, the memory order m of 
the encoder is defined as 


I max { i | G . ^ 0 } 

0<i <o° 1 (12) 

oo if no such i exists. 

All encoders with m = 0 are block encoders. Therefore con- 
volutional codes include block codes as a special case. 
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Let u be the least value of u such that m(u) is a 
max 

maximum if m(u) is finite for all u > 0. Otherwise let u 

max 

be the least value of u such that m(u) is infinite. Then the 

number of encoded digits in those encoded blocks inclusive 

between the first and last encoded block which depend on the 

•Mmfi-ii block of inout dicrits is defined to be the encodincr 
max 

constraint length n^. For both general and fixed convolu- 
tional encoders of memory order m. 


n = N (m + 1) . 

■T* 


(13) 


For time-varying encoders of memory order m, the gener- 
ator matrix can be rewritten as 


G = 


g 0 (°>% (0 >£ 2 (0) 

... 5 ^( 0 ) 

0 

£ Sq (1)^(1) 

*•* %n-l (1) 

G (1) 
nn 

o a Gq ( 2 ) 

••• 2 m-2 (2) 

G _ ( 
-m-1 


G (2) 

“TO 


(14) 


and for fixed encoders of memory order m, the generator matrix 
becomes 


G = 


% 

% 

-2* * * 

G 

“TO 

£ 

lo 

• 

• 

9 

1 


Q 

2 0 

G, • • • 

G . 

G 

0 ... 

(15) 


-1 

~m-l 

“TO 


0 

0 

G_ • * • 

G 

G i 

G • • • 



~0 

m-2 

—m-1 




If 


G q (u) = [l K : & Q (u)J , 0 < u <oo 


(16) 
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where I K is the K x K identity matrix and Qq(u) is a K x 
(N - K) matrix of elements from GF(q) , and 

Ghtu) = j^ 0 K ! fii <u>] / 0 ^ u<oo , 1 ^ i < oo , (17) 

where is the K x K all-zero matrix and Q^u) is a K x 
(N - K) matrix of elements from GF(q>, then the encoder is 
said to be in canonic systematic form. An encoder is said 
to be in systematic form if the output terminals can be re- 
numbered so that the resultant encoder is in canonic system- 
atic form. For systematic encoders some set of K output 
sequences are reproductions of the K input sequences, and 
for canonic systematic encoders, the transmitted sequence y 
can be written as 


Y =[x 0 , J2 0 ' ... ] 


(k) (K+l) 


r (i) 

v (K) 

L x o 

. . .X Q 

__ (K) 

_ (K+l) 

X 1 

P 1 


(N) „ (1) 


0 




(N) 


...] . 


(18) 


(19) 


where is an (N - K) -dimensional row vector called a 


parity vector , and p^ 


<J> H 


is called a parity digit , 0 < i <oo / 


K + 1 < j < N. 


A fixed code is in canonic systematic form if 



^0 “ 

1 1 

H 

* 

•• 

lo 

0 

1 i 

(20) 

and 

G. = 

“l 

°K : qJ , 1 < i < oo . 

(21) 

This 

is equivalent to requiring that 



G(D) 

= [ Z K = 2 <D >] ' 

(22) 


where 
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2(D) = &0 + 2-jD + 2 2 d + ... • (23) 
In this case the transform y(D) of the transmitted sequence 
can be written as 


Z(D) 




( 1 ) 


(1C) 


(K+l) 


(N) 


(D) X ' (D) , p'~ ' (D) p ( D ) 


]. 


(24) 


where 


(j) 


(j) 


(j>. 


< __ . 'j/ \ j / 

(D) = p Q + p x D + p 


(J>«2 


(25) 


tli ( i ) 

is the transform of the j parity sequence Pq' j; 


(j) 


(j) 


, ..., K + 1 < j ^ N. 

For a time-varying convolutional encoder, if 
Ghfu) = G ± (u + T) , 0 ^ i <°o, 0 < u<co , 


(26) 


the encoder is called periodic with period T. Clearly a 
fixed encoder is a periodic encoder with period T = 1. 

1 

The most familiar convolutional encoders are the R = — 

N 

fixed encoders. In this case the first row of the generator 
matrix G is called the generator , and is labeled 2* Figure 
1.2 shows an example of what Forney calls the obvious realiza - 


tion [6] of a binary R = 1/2 canonic systematic fixed con- 

f 1 + D + 

L 1 1 + D 


Note that 


volutional encoder with G(D) 
the encoder is realized with two memory elements even though 
the memory order of the encoder is infinite. When G(D) con- 
tains only polynomial elements the number of memory elements 
required in the obvious realization is equal to the memory 


order of the encoder. However, a rational function 


P(D) 

Q(D) 

requires only max j degree [p(D)] , degree [Q(D)] j memory 
elements in its realization. The feedback used to realize a 
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rational function accounts for the infinite memory order of 
the encoder. 

It is usually desirable to eliminate feedback in the 
encoder so that an encoding error caused by an equipment mal- 
function will not propagate (cause other errors) throughout 
the transmitted sequence. Feedback can be eliminated by 
multiplying each row of G(D) by the least common multiple of 
the denominators of its generator functions resulting in a 
matrix G' (D) with only polynomial elements. The resulting 
encoder is equivalent to the original encoder. For example, 
the encoder given above has G' (D) = £l + D 1+D+D^J. 

An input x(D) = I x^ (D) I = x(D) to G(D) produces the same 

x(D) 

codeword as an input X"TT!j to G' (D) . The obvious realiza- 
tion of G' (D) is shown in Figure 1.3. Note that the poly- 
nomial encoder has memory order two but is not canonic system- 
atic. 

C. Review of Algebraic Decoding 
Let 


H 


HqIO) 

h 2 (0> 


HqU) 

%<!) 



be a semi -infinite matrix and 


( 27 ) 
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H^(u) 


Xl 

Xz 


( 1 ) 

(1) 


(u) 

(u) 




(u) 

(u) 


(N) 

Xi 

^ (N) 

Xz 


(u) 

(u) 


(28) 




n <t,N-K 


(N) 


(U) 


be an (N - K) x N matrix of elements from GF(q) , 0 < u, ■t 
<°°, such that 


rank [Hq^J = N - K, 0 < u<oo , (29) 

G H T = 0, (30) 

T 

where H is the transpose of H, 0 is a semi-infinite all- 
zero matrix, and each sequence hg^-^ ( u ) ' ^li^ ( u ) ' ^2i^^ ' 

1 < i < N - K, 1 < j < N, 0 £ u < 0 °, is realizable. 

These sequences are called pari tv- check sequences and H is 
called a paritv-check matrix . 

The parity-check matrix for fixed convolutional encoders 
is written as 


H 


Bo 

% 


0 




/ 


(31) 


where 
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% " 


. (1) 

■u (2 > 

( 

\l 

^Xl 


- (1) 

(2) 


h t2 

h X2 

h l2 


(N) 

(N) 


, (1) . (2) - (N> 

n X,N-K n X,N-K *** n t,N-K 


,0 £ Koo, , (32) 


rank [h q ] = N - K, (33) 

and each parity-check sequence ^i^'***' 

1 < i < N - K, 1 < j < N, is realizable. It is often con- 
venient to introduce an (N - K) x N matrix H(D) , where 

(N) 


H(D), = 


H 1 (1) (D) 
h 2 (1) (D) 


( 1 ) 


V 2 > (D) 


H, 


(D) 


H 2 ( 2) (D) . . . H 2 (N) (D) 


( 2 ) 


L^-K (d) ^-k (D > • • * **-: 


(N) 

K 


(D) 


(34) 


and 


H, (j > (D) = h_. + h n j (j *D + h OJ (jJ D 2 + 


(j> 


h 


li 


(35) 

(j) 


1 Oi “li " 2i 

is the transform of the parity-check sequence h Qi 

h 2i ^^ , . . . , 1 < i < N - K, 1 < j < N. Each (D) is 

called a parity-check function . Then equations ( 30) , the 

parity-check equations , can be written in D-operator form as 

T 


G (D) H (D) = 0 , 

where 0 is the K x (N-K) all-zero matrix. 
For any transmitted sequence y 
y H T = x § n; T = x 0 = 0 , 


(36) 


( 37 ) 


where 0 is a semi-infinite all-zero vector. For fixed en 
coders equation (37) can be written in D-operator form as 
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y(D) H T (D) = x(D) G (D) H T (D) = x(D) 0 = 0, (38) 

where in this case 0 is the (N - K) -dimensional all-zero 
vector. Therefore every transmitted sequence is in the null 
space of H T . However if rank < N - K, the encoder has 

no zero-delay inverse [5] , and the null space of H contains 
other sequences besides the set of codewords. 

Note that 



H_. 

—0 

Si 

S 2 

S 3 

s 4 

“5 * * * 



H* = 

0 

So 

% 

s 2 

S 3 

s 4 • • • 


(39) 


0 

0 

S 0. 

% 

s 2 

h 3 ... 




- 


• 



N - 

K 


xs a generator matrxx 

for 

an 

R 

— ; encoder. 

This encoder 


is said to be dual to the encoder whose generator matrix is 
G. For fixed encoders, H(D) is the matrix of generator func- 
tions for the dual encoder. 

Figure 1.4 shows a simplified picture of a channel with 

additive noise. Let the noise be represented by an error 

(N) (1) 


sequ en ce e ^Sq'%' * * * ] = [ e o ^ * * * e 0 


•ei (N > 


( 1 ) 


• e. 


(N) 


...] . 


(j) 


where e.. ' J ' e GF(q) for all i and j. The 


is e(D) =|j= 


( 1 ) 


(D) , 

where e^ (D) = e Q ^ + e^-^D + e 2 ^D 2 + 


(N) 


transform of the error sequence is 
(D)] , 

1 < j < N. Similarly the received sequence r = |r^, r^, r ^ • 1 

[ /it, /..t / i \ /..t /it /._t *- *i -J 

3 


(1) 


(N) _ (1) 


(N> 


0 *** ■‘‘O ‘1 — ■‘‘1 

transform of the received sequence i 
(N) 


(1) ^ (N) 

2 * • * r 2 


r 0 * * an< ^ 


|r (1) (D) 


(D) 


is r(D) = £i 

Since the noise is additive, the received sequence 
is the modulo-q sum of the transmitted sequence and the error 
sequence, i.e.. 


>]■ 




I? a 


r = Y. + ® • (40) 

In terms of transforms, equation (40) is 

r (D) = y(D) + e(D) . (41) 

Clearly, for all i and j, 

r^ ^ f if and only if e^^ ^ 0. (42) 

Thus e^-^ ^ 0 corresponds to an error in transmission, 
whereas e^-^ =0 corresponds to a correctly transmitted 
digit, 

A common form of algebraic decoding computes the syn- 

, r i r (l) (n-k) (i) 

drome sequence s_ =1 .Sq, s ^, . . .1 = I s Q • • • s q S 1 * * * 

s^ (N K) ^ J at rece i ver from the parity-check matrix as 
follows? 

s = r H T . (43) 

From equations (37) and (40) it can easily be deduced that 
s = e H T . (44) 

In terms of transforms .s(D) = ^s^(D),..., s^ N_K ^ (D) J , 

where s (j) (D) = s Q ( J> + D + s 2 (j) D 2 + ..., 1 < j < 

N-K, and for fixed encoders equation (44) can be written as 
s(D) = e(D) H T (D) . (45) 

Therefore the syndrome depends only on the errors and not on 
the particular codeword transmitted. Any decoding function 
which estimates the error sequence from the syndrome is 
called a syndrome decoder . Equation (44) and (45) are re- 
ferred to as the syndrome equations . 

It is often necessary to write the encoding equations 
over only one constraint length of transmitted digits. If 
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z = ^o' ~ 1 ' ~2' *’*] ' tlien l et ^ denote the truncated 
vector |^Zq/ z-^/ . . . , z^ J . Then the encoding equations for 


the first constraint length are 


[*]»- w ra rso<o) %«»••• v°> ‘ 


£ ... G q (m) 


M™ [S] 


= [S5], - <«> 

where I’g'I indicates that G has been truncated after 
L — J m “ 

(m + 1)N columns. 

In syndrome decoding it is usually assumed that the 
error digits at time unit u, are estimated by looking at 

the syndrome digits .s^, from time unit u 

through time unit u + m, 0 < u <°°. mis called the decoding 
memory , and n A = N(m + 1) is called the decoding constraint 
length . Therefore the first constraint length of syndrome 
equations can be written as 


[3] - = [ S ] 


Hq T (0) h^ t (0) ... BJJ (0) 

o Ho T U) ... H^U) 


H 0 T (m) 
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(50) 

(51) 


Tl T 

where [H J indicates that H has been truncated after 
m 

(in + 1) (N - K) columns, in is often chosen as 


m 


max max { i|f§. (u) ^ 0 | 

0<u<oo 0<i<°° 1 


oo 


if no such i exists. 


For fixed encoders equation (52) reduces to 


(52) 


max { i|fi. f 0 [ (53) 

0<i <°° x 

° 0 if no such i exists. 

Choosing m in this way corresponds to defining the decoding 
constraint length n A as the number of error digits inclusive 
between the first and last blocks of error digits which af- 
fect the syndrome at time u + m, where u is the least 
J max max 

value of u which maximizes (52) . 

For canonic systematic encoders, the generator matrix 
can be written as 



_I K : 20 (0) 

• °K T -1 (0) 

°K 

: Q 2 (0) : ...- 


0 . 

I K S °0 (1) 

: 0 

K 

: Q (1) : ... 


0 

0 

t k 

? Qo(2) : ... 

••• 

(54) 


Clearly, it follows that 
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VV 0)S0 K : V 0):0 K^2 (0) : •* 


0 

0 


T : Q (1) ;0 :Q (1) 
X K -0 K -1 

0 I k s2q(2) 


“Qq (0> “Qi (0> -Q 2 (0) — 


10 0 
N-K N-K N-K 


-Qo(l) -Q^D-.. 


X N-K °N-K * * * 


0 


-Q 0 (2) 


l n-k* * * 


where I N _ K is the (N - K) x (N - K) identity matrix and 
0 N _ K is the (N - K) x (N - K) all-zero matrix. Since 


=0 , 


(55) 


rank [ J N-K 1 -2 q T( u) ] = 


N-K, 0 < u < 00 , 


(56) 


H = 


“Q0 (°) : I n _k S. 

-a i T( 0 ): °N-K ! V' 1 > :I N-K 

~-2 s0 N-K ! "2l :0 N-K 


0 

0 


-Q n (2) : I 

N-K 


(57) 


is a valid parity-check matrix for the encoder with genera- 
tor matrix G. For fixed canonic systematic encoders 


G = 


X K * — 0 1 °K ‘ °1 ! °K 1 ° 2 = ••• ' 

0 * Ij^ • Q 0 • • Q j • • • * 

- - t k : : * * * 

• * _ 


( 58 ) 


and 
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H = 


°o s 

■'■H-K 


0 


0 

T 


T 




fii s 

°N-K ! 

-°o 

9 

9 

I N-K 

0 

T 


„ T 



^ T 

q 2 : 
• 

°N-K ! 

9 

~ -1 
• 

9 

9 

°N-K S 

• 

- £o : 

9 

9 

9 

• 

9 

9 

• 


9 

9 

9 


N-K 


(59) 


is a valid parity-check matrix. Therefore if the decoding 
memory m is chosen according to equation (52) for periodic 
encoders or equation (53) for fixed encoders, it can be seen 
that m = m for systematic encoders and hence n A = n A * H 
will be related to G for non-systematic encoders in Chapter 
III. 

The two commonly used modes of algebraic decoding have 
been termed feedback decoding and definite decoding by 
Robinson [7] . Although usually used with syndrome decoding, 
these two decoding modes can be used with any algebraic de- 
coding technique. 

If z = ^ z 1# z 2 , . . .J, then let denote the 

h,h+i 

doubly truncated vector ^z^, J * T ^ en the 

syndrome equations from time unit u through time unit u + m 
can be written as 


l>] 


u,u+m 


l>] - - 

u-m,u+m 


T _ 

H— (u-m) ... 0 

•m 


*T * rn 

H~ (u) ... H — 1 (u) 

— 0 — m ' ' 


0 ... Hq T ( u+m) 


( 60 ) 
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In feedback decoding, at time unit u + m all blocks of error 
digits e Q , e^, . . . , ^ up to time unit u are assumed to 

have been decoded correctly and are "fed back" and sub- 
tracted out of each syndrome equation in which they appear. 
Therefore equations (60) become 


[a] =[e] 

u , u+m u , u+m 


iji 

H 0 T (u) . . . H m (u) 


0 ... Hq T ( u+m) J 


(61) 


and e^ is estimated from equations (61) . In feedback de- 
coding, the decoding memory m is always chosen to be m and 
the feedback decoding constraint length n pD = N (M + 1) = n A * 
In definite decoding, the error digits up to time unit u 
are not "fed back" and is estimated directly from equa- 
tions (60) . The definite decoding constraint length n^ is 
sometimes chosen to be the number of error digits which can 
affect equations (60) for u > m, i.e., n DD = N (2m + 1), but 
it is often useful to choose it otherwise, as will be seen 
in later chapters. Figures 1.5 and 1.6 show a syndrome de- 
coder operating in the feedback and definite decoding modes, 
respectively, for the binary fixed R = 1/2 encoder with G(D) 
[1 1 + D] . 




II. Distance Definitions for Convolutional Encoders 

In this chapter a number of different distance measures 
will be defined for convolutional encoders. Distances be- 
tween codewords in a convolutional code are closely con- 
nected with the probability of decoding error. For instance, 

it is well known [3] that for an encoder with feedback de- 

d FD " 1 

coding minimum distance d^ (formerly called d m ^ n ) / 2 

errors within a feedback decoding constraint length are guar- 
anteed correctable by an algebraic decoder operating in the 
feedback decoding mode, where [ij is the largest integer 
less than or equal to I. Hence it is usually desirable to 
design an encoder with good distance properties. For con- 
venience, in the remainder of this thesis only binary codes 
will be considered, although many of the results apply to 
codes defined over larger alphabets. 

A. Feedback Decoding Minimum Distance 

The standard definition of distance for convolutional 
encoders, feedback decoding minimum distance , dp , will now 
be generalized to time-varying encoders. 


Definition 2.1 


d = min min 

FD 0 <u<°o x ^x 1 

~"U u 


y [22] 


u,u+m 


M , = [S'] , 

u— 1 L J u— 1 


[x 1 G] ) = min min c^( [x G ] 

L u,u+m 0<u <oo x^x^ 


rx’Gi ^ ) = 
l J u+nr 


[2] , = [S'] 

u-1 u— 1 


min mm 

0 <u<°° x £x 1 
-u u 


y [2 s] 


[x'G] ) = min min 

u+m 0<u <oo x ^x 1 
— u — a 


24 
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[y] / [y‘] ) / where <!!(•,•) denotes the Hamming 

H u+m u+m H 

distance between the two arguments and the minimization is 
over all x and x' with x ^ x '/ 0 < u <°°.| 

The second equality in definition 2.1 follows from the fact 

that d H ( [ x G ] , [x'G] ) = 0 if [x] = [x 1 ] , 

u-1 u-1 u-1 u-1 

and hence the Hamming distance between codewords is not 

affected. The third equality follows from the fact that all 

pairs of codewords with [x] ^ [x 1 ] were already in- 

u— 1 u-1 

eluded in the minimization for some smaller value of u and 
hence cannot change the minimum. 

For periodic time- varying convolutional encoders with 
period T, definition 2.1 reduces to the following definition 


of ^D* 

Definition 2.2 


^FD 


min 

0<u<T 


nun 

x ^x 1 
- u — u 


[a 2] 


u+m , 


[x'G] ) = min min [y] / [y'] ) . I 

u+m 0<u<T x,,^x 1 u+m u+m 

“11 — u 

For time-varying encoders with finite memory m < 00 , the 

pattern of the first K(m + 1) rows of G must be repeated some 

where in G. Suppose this pattern is repeated beginning at 

st 

row KT (the (KT + 1) row of G) , where T is some positive 
integer. Then a periodic encoder with period T can be formed 
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by reproducing periodically the first KT rows of G. It can 
easily be seen from definitions 2.1 and 2.2 that for the 

periodic encoder is at least as great as d^, for the original 
encoder. Similar arguments to the above can be given for 
each of the distance measures defined in this chapter. Hence 
as far as distance properties are concerned it is of no value 
to consider non-periodic time-varying encoders. Therefore all 
time-varying encoders will be considered to be periodic. 

Finally, d^ will now be defined for fixed encoders. 


Definition 2.3 



mm 


^0 


* X 


‘0 


djj( [x G] , [x'G] ) = min 

m m x~ j 


*0 


/ x 


0 


v [z] . [z'] ) -I 

m m 


Because of the linearity of convolutional encoders, defin- 
itions 2.2 and 2.3 can be simplified. 


Theorem 2.1 


^FD 


= min 


mm 


0 < u < T x / £ 


w ( [ x G ] ) = min 


H 


— u 


u+m Q < u < T 


min w ( [ y ] ) , where w (•) denotes the Hamming weight 

v ^ H 

^U ~ 


u+m 


of the argument. 


Proof Let 2S and x' be any two information sequences with 

^ x ' such that [y] = [x G ] and [y 1 ] = [x'G] 

” u u+m u+m u+m u+m 

and let x" = x + x 1 • Then x " ^ 0 and [y"] = [ x"G] 


u+m u+m 

[ (x + x') G ] = [x G + x'G ] = [x G] + [x’G] 

u+m u+m u+m u+m 

fyl + f y'l since the encoder is linear. Therefore, 

u+m u+m 

v M ' [x 1 ] ) = w H ([y] + [£'] ) = w ( [y"] ) 

u+m u+m u+m u+m M u+m 
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and the theorem follows immediately. | 

For fixed encoders, Wozencraft and Reiffen [3] proved 

that 


= min , ft w h ^[ 2 £G] ) = min w([y] ). (62) 

Xq ^ H m Xq ^ £ H m 

A generalization of definitions 2.2 and 2.3 will prove 
useful later in the construction of convolutional encoders 
with high d^* 

Definition 2.4 The order j column distance , d ^ / of a period- 
ic encoder is given by 


d . = min min d ( [ x G] , [x'G] ) = min 

3 0 < u < T ^ u ^ u+ j u+ j 0 < u < T 


min , d ( [y] , [y] l, j = 0,1,2 and d^ = 

x / x ' H u+j u+j 

u U J J 

lim d.. | 

j — f oo 


Definition 2.5 The order j column distance , dj , of a fixed 
encoder is given by 


d , = min d ( [ x G ] , f x'G 1 ) 

J x~ / H L- 1 L J 


— 0 ' “0 


i *o * ^o 1 


j = 0,1,2,..., and = lim d . • I 

j— »oo 3 

Some simple properties of the column distance will now be 
collected. The first property follows as a direct consequence 
of definitions 2.2 through 2.5, and applies to both periodic 
and fixed encoders. 


The next property can be proved by a slight modification of 
the proof of theorem 2.1. 
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d . = min min 

J 0 < u < T x / .0 


u+j 0 <; u < T 


.( [y] ) for periodic encoders and d. = min 


Xq^O 


w ( r x G ] ) = min w tt([x] )for fixed encoders, 

H J 2„^fi H j 


j = 0,1,2, 


Let a 


(1) (2) (N) . . . 

J 0i g 0i ••• g 0i « U 9 U W 


be the i row of the generator matrix of a fixed encoder, 

1 < i < K. contains N generator sequences and is called 

the 1 th generator. Then let j^gj = £g oi (1) . . .g 0 .[ N) . . .g . . ( 


•,r ] 


be the (j + l)N-tuple consisting of the first (j + 1)N 


entries in g^. This notation simplifies the statement of the 


next property for fixed encoders. 

Property C3 d . < min min 

J 0 < u < T 1 < i < K 


W H* [ g 0i • • • 


(u) . . . g . . 1 ; (u) . . . g 




) for periodic encoders 


and d. < min 


1 < i < K 


M j 


) for fixed encoders, j = 0,1,2, 


Proof The proof will be given only for fixed encoders in 
order to simplify the notation. Let ■t be the integer, 1 £ £ < K, 


Then 


such that w H ( £g^J ) < w H ( £g^J ) , 1 £ i £ K, i ^ Then 
let x« =fO...O 1 0...0] , X, = x^ = ... =0, where the 1 is 

_iQ >- 12 

in the b position. Therefore [x G] = 1 and hence by 

j L -> j 

property C2, d. < min w ( |g.| ), j = 0,1,2,... . | 

J l^i^K L 1 J j 


J 1 £ i £ K L -I j 

d. ^ d. , , j = 0,1,2,..., for both periodic and 
J J+l 


fixed encoders. 
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Proof 


[*] 


U+ J+1 u+j u+j+1 


But min 
0<u<T 


min w ( Ul ) < min 

x H U+j 

— U — J 


mxn 

0<u<T x . 

a 


i+x] 

A W H<[W u+J 'Wx]> 


min min w ( [y] 

0<u<T u+j+1 


) . Therefore d . < d . , , , 

J J + 1 


j = 0,1,2... . The proof for fixed encoders is similar and 

hence is omitted. ] 

B. Definite Decoding Minimum Distance 

A definite decoding minimum distance , d^p, and a definite 
decoding constraint length , n^, will be defined such that 
^DD ” 1 

— — — errors within a definite decoding constraint length 
are guaranteed correctable for codes being used with definite 
decoding. It is convenient to distinguish non-systematic en- 
coders from systematic encoders. 

1. Non-systematic Encoders 

Robinson [7] suggested the following definitions of 
and These definitions will be restricted to non-systema- 

tic encoders and the definition of d^ will be generalized 
to periodic encoders. 

Definition 2.6 The definite decoding constraint length n^ 
is the number of error digits which can affect the syndrome 
equations from time u through time u + in, u > m, i.e., n^ = 
(2m + 1) N. | 

Definition 2.7 dp^ = min min 

2rn < u < 2m + T ^u ^ ^u* 

d ( [x G] _ __ ,[x'G] _ _ ) = min _ 

H u-m,u+m u-m,u+m 2fiT < u < 2m + T 


min 

x ^ x 


V 




U 


u-irt/U+m 


_ . [£'] _ _> 


u-m,u+m 
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where the minimization is over all x and x 1 with x £ x ; , 

— — — u 

2m < u < 2m + T. | 

For fixed encoders, definition 2.7 reduces to Robinson's 
[7] definition of d^. 


Definition 2.8 


= mln [x'G] _ _) 

5C ««. ^ t in / 5m 


'2m ~ 2m 


= min 

x„— x 
~2m _ 2m 


V [2] _ [z'] _ _ 

m , 3m m , 3m 


A similar argument to that used to prove theorem 2.1 
results in the following simplification of definition 2.7. 


Theorem 2 . 2 




= min 

2m< u < 2m + T 


min w„( f x Gl 

^#0 H L J 


_> 

u-m,u+m 


= mm 

2m < u < 2m+T 


min w ( [ y ] _ _ ) • 

^ Q. u-m,u+m 


^u 


Robinson [7] has also shown that for fixed encoders definition 
2.8 can be simplified to 


^DD 


= min 




w H ( [X G] ) = min 


m,3m x — 5 

2m r ~ 


w H < [2] _ J - 

m , 3m 


(63) 

2. Systematic Encoders 

For canonic systematic encoders the syndrome equations 
from time unit u through time unit u + m can be written as 
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ui _ = ui _ „ 

u,u+m u-m,u+m 


Q— (u-m) 

— m 


N-K 

Q (u) 

“0 

• • 

^N-K 


0 


0 


Q - (u) 
— m v ' 


°N-K 

Q q ( u+m) 

• « 

"‘‘N-K 


(64) 


where each Q^(u) , 0 <: i < m, 0 < u < °°, is a K x (N - K) 
matrix of elements from GF(q) . Hence the last (N - K) digits 


m e 


—u-m' “u-m+1 


, e u _ 2 ' and ^u-i' i-®*' the digits in 


the parity positions of all the error blocks previous to 
time u, do not affect the syndrome equations. Therefore de- 
finitions 2.6, 2.7, and 2.8 can be modified for systematic 
encoders as suggested by Massey [ 8 ] . 

Definition 2.9 tlie num ker of error digits which can 

affect the syndrome equations from time u through time u+m, 
u > in, is N(m + 1) + Km for systematic encoders. | 

min 


Definition 2.1.0 

O] 


^dd = min 

2m < u < 2m + T 


x x 1 


y[i 


~ ' C— — ] — — /fx’G'l 1) 

u-m,u-l u , u+m J |_ u-m,u-l L J U/U+ m 


for periodic systematic encoders. 
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Definition 2.11 


^DD = mln 


s 2m *-■>* 


H 


2m 


<rw - - - 

j_ m,2m-l 


[x G ] _ _|,r[x'] _ _ , [x'G] _ _ 1) f° r fi xe< ^ sy sterna™ 

2m / 3mJ [_ m, 2m-l 2m,3mJ 

tic encoders. | 

Again, a similar argument to that used to prove theorems 
2.1 and 2.2 results in the following simplification of defin- 
ition 2.10. 

Theorem 2.3 d^ = min min w T 

2m £ u < 2m+T 3^ ^ 0 

[a a] _]) . I 

u , u+mj 

For fixed encoders, Massey [8] has shown that definition 2.11 
can also be simplified to 


*H <T[S] - - 

u-m,u-l 




= mm 




w. 


H 



m, 2m- 1 



(65) 


As discussed in Chapter I, if the decoding memory is 
chosen according to equation (52) for periodic systematic en- 
coders or equation (53) for fixed systematic encoders, then 
the decoding memory is the same as the encoding memory, and !n 
can be replaced by m in definitions 2.9 through 2.11, in 
theorem 2.3, and in equation (65). 

It can easily be seen from definitions 2.2, 2.7, and 
2.10 that for periodic encoders 


^DD ~ ^FD 7 


( 66 ) 


a result which is well known [7] for fixed encoders. 
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For each definition of in Section II. B, an argu- 
ment similar to Robinson ' s [7] for fixed non-systematic 

[ dpD - 1 

encoders can be used to show that 2 errors within 
a definite decoding constraint length are guaranteed correct- 
able by an algebraic decoder operating in the definite de- 
coding mode. 

C. Free Distance 

For decoding schemes such as sequential decoding, in 
which the decoder is not constrained to consider only one 
constraint length of received digits while attempting to de- 
code a particular block of transmitted digits, but may search 
over a much longer portion of the received sequence, dp D and 
dpQ are no longer meaningful. Consequently a different dis- 
tance measure, called the free distance bv Massev [9] , and 
studied by McEliece and Rumsey [10] and Neumann [ll] , will 
be considered. Free distance is defined over the entire en- 
coded sequence and hence is appropriate for a decoder which 
makes its decisions on the basis of the entire received se- 
quence. 

Before defining free distance, a more general distance 
measure will be introduced. 


Definition 2.12 The order j row distance , r , of a periodic 
encoder is given by 


r . = min mm 
J 0£u<T 


V[M . - 


-[*■] 


j = 0,1,2,..., and r 


lim r . , where 0 represents an 

j CO J 


infinite concatenation of all-zero K-tuples. 
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Definition 2.13 The order i row distance, r., of a fixed 

— - J — J 

encoder is given by 


r . = min 


H 


rr -i O0 1 

[x] , 0 G, 

oo“l 

[X'] , 0 ] 

j J 

j J 


2, . „ e , and = lim r.-l 

27 — > OO 

Some simple properties of the row distance will now be 
given. The first property can be proved by again slightly 
modifying the proof of theorem 2.1 


Property Rl r . = min 


mm 


J 0<u<T 3^0 


, r r _ oo" 

w ( [x] , 0 G) 

L u+ j 


[X] =0 

U“1 


for periodic encoders and rj = min w^^x] , £°Jg) for 

fixed encoders, j = 0,1,2,... .j 

As was seen in property C4, the column distance dj cannot de- 
crease with increasing j. Just the opposite is true for row 
distance, as will now be shown. 

Property R2 r^ > r j + -j/ J = 0,1,2,..., for both periodic and 
fixed encoders. 

Proo f Let Y = J1 fxl , 0, 


l[“> 


ocn 


u+j+1 


G| [x] = 0, 3 ^ t Ojand 


Y 1 = 


tains 2' 


j[M .. 0°°]s I [X] =0, 

( L u+ j J u-i ) 

JC(j+lJ , _K . . . 


Then Y con- 
IC j , K 


(2 -1) elements and Y' contains 2 (2 -1) 

'[*] 


elements. Clearly Y' C Y. Therefore min w ( 

V 2 

[x] =o 

U-l 


U+j+1 


oo 


0 


G) < min 

V 2 

[X] = 0 

U»1 


w ([[x] , 0°° 1 G) 

L u+ j 


Hence r \ < r , 
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j = 0,1,2,..., for periodic encoders. The proof for fixed 
encoders follows in a similar fashion. | 

Property R3 r . < min min w I g (u) . . . 

3 0 < u < T 1 < i < K L U1 

g Qi ^ (u) g.^^ (u) * • -g 1;L (N) (u) . . . ] for periodic encoders and 

r. < min wl, ( cj . ) for fixed encoders, j = 0,1,2,... . 

3 1 < i < K H 1 

Proof The proof will be given only for fixed encoders in 

order to simplify the notation. r n = min w (cl) . 

u 1 < i < K H 1 

Therefore property R2 implies that r . < r = min 

3 u 1 < i < K 


) f J ““ 0/1/2/. • j 

ri X 

Now a simple result relating the column distance d^ to 
the row distance r^ will be given. 

Theorem 2.4 d < ^ < ... < d^ < r^ < ... < ^ ^ r Q for both 
periodic and fixed encoders. 

Proof Again the proof will be given only for fixed encoders 


in order to simplify the notation involved. Let y = 


=ro] / 

,7 L n J 


Cf] G and y' = [x G] . Then y = [y 1 , y J+1 ' Y j+m] ' 

j = 0, 1, ;2, . . . 


Hence w R (Z’) ^ w H (y) and r^ 


lim r . = lim 

oo J i — ► oo 


min 


Xq / 0 


jr°° J j 

tr ( [ f X'l , 0°° 1 G) ^ lim min w ( [x G] ) = 

H L j J J-°° / fl H j 


lim d. = <1^. Clearly property C4 implies that d. < lim 
i— »oo J -* i 1 —* 


oo 


d . , = d and property R2 implies that r . > lim r . , = r^ 

J J g 1 — * OO J 

for all finite j . The theorem then follows immediately from 
these two properties. | 
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Now the free distance, d^^g, will be defined as the 

minimum weight encoded sequence such that x ^ 0. 

Definition 2.14 ^free = m ^ n W H (2£ S) for both periodic 

x ^ 0 

and fixed encoders. j 

Note that is a property of the code itself. 

Some properties of 3p REE will be presented next. 

Property FI dp REE = < co ^ or fixed and periodic encoders. 

Proof For fixed encoders, d = lim min w„( [x g] ) = 

j-~Xo^O Hl , j 

min w„(x G) . Now let n be the smallest integer such that 

So / o H 

x^ f 0- Then let x' be the information sequence whose trans- 
form is x' (D) = D n x(D) > where x(D) is the transform of x. 

Clearly x' ^ 0/ x' is casual, and w (x'G) = w (x G) . Hence 
0 H H 

every codeword produced by a non-zero input sequence x has 
the same weight as the codeword produced by the input sequence 
x' with Xq ^ 0. Therefore min w^(x G) = min w^(x G) and 


2i0 / 0 


H 


x / 0 


H 


dpREE = <*». T ^ e P ro °f for periodic encoders follows in a 
similar fashion. | 

The second property follows directly from property Cl, theorem 
2.4, property FI, and definition 2.14. 


w 

H 


<[' 


( 1 ) 


f 0i 


en- 


P roper tv F2 cL < d^ < min min 

FD FREE 0 <u<Tl<i<K 

(u)... g Qi ^ N ^ (u) g 1± ^ (u) •••g li ^ (u) •••] ) for periodic 

coders and d^ < <3p REE £ min f° r encoders. 

1 ^ i ^ k 

The third property shows that dp REE i s unchanged if the set 
of allowable input sequences is expanded to include all 
rational sequences. Hence this property serves as an alter- 
nate definition of <3p REE . 
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Property F3 ^ F ree = m '*' n w R (y) / where the minimization 

is over all rational input sequences x / 0, for both per- 
iodic and fixed encoders. 

Proof Let x be any non-causal rational input sequence and 

let n be the least integer such that x £'0. Then let x' be 

~~n ' ~ — 

IcT 

the input sequence whose transform is x' (D) = D ^(D) # where 
x(D) is the transform of x, T is the period of the encoder, 
and k is the least positive integer such that kT > - n. 

Clearly x' is causal, x' j* 0, and w R (y') = w R (y) , where y' 
is the codeword produced by x' and y is the codeword produced 
by x* Hence every codeword produced by a non-causal input 
sequence x has the same weight as the codeword produced by 
the causal input sequence x' • Therefore the set of input 
sequences over which the Hamming weight of codewords is mini- 
mized in the definition of <ip REE can be expanded to include 
all rational sequences without changing <3p REE * I 
The fourth property applies only to fixed polynomial encoders, 
i.e., to fixed encoders whose matrix of generator functions 
G(D) contains only polynomial elements. 

Property F4 For all fixed polynomial encoders with a feed- 
forward inverse, d. „„ = d, = d = ••• = d = r = ... = 

FREE j j+1 oo oo 

r . m for some finite i . 
j -m J 

Proof A fixed polynomial encoder has a feedforward inverse 
if there exists an N x K matrix of polynomials G - ^ (D) such 
that G ( D) G 1 (d) = D^I.,/ where L is called the delay of the 
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inverse. Massey and Sain [ 12 ] have shown that L < Km, where 

m is the memory of the encoder. Necessary and sufficient 

conditions for a fixed encoder to have a feedforward (i.e., 

polynomial) inverse are given by Olson [ 13 ] . Clearly, if 

G(D) G“ 1 (D) = D L I„, then s(D) G(D) G _1 (D) = y(D) G -1 (D) = 

K 

D x(E>) for any input sequence whose transform is x (X>) . 

In minimizing w„(x G) over all input sequences x ^ 0, 

it is not necessary to consider any input sequence with a 

string of m or more blocks of 0,' s in it since following such 

a string with additional non-zero blocks can only add to the 

weight of the codeword. Also, property F2 implies that <ip REE 

can never be more than N (m + 1) , the maximum number of 1 ' s 

in any generator, i.e., cl „„ is finite. 

FREE 


Now let M be the maximum degree of the polynomials in 


-1 


G (D) . Clearly, if y contains a string of (m + M + 1) or 
more blocks of O' s, x must contain a string of m or more blocks 
of 0's. Hence for each input sequence x with 0 which is 

capable of producing the minimum free weight codeword, y L ^ 0 
and each successive string of (m + M + 1) blocks of encoded 
digits must contain at least one non-zero block. Therefore 
the first (Nm +N-1) (M + m + l)+L + l blocks of encoded 
digits must have weight at least N(m + 1) for all encoded 
sequences capable of producing the minimum free weight code- 
word. This implies that d_^__ = d = min w (x G ) = 

1? Kriii U 

2o * 2 

d (Nm + N - 11 (M + m + 1) + L 3113 that 'W = 'to = mln , „ 

20 ^ 2 
Hence from 


W H^“ r (Nm + N - 1) (M + m + 1) + L - m* 
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theorem 2.4, = dj = d j+1 = ... = ^ = ... = rj_ m 

for some finite j . \ 

Property F4 is a very crude bound on the length of in- 
formation sequence needed to produce the minimum free weight 
codeword. It is conjectured that this result can be strength- 
ened considerably by more detailed arguments. Neumann [ll] 

has suggested that the true bound is m, i.e., that d = r . 

FREE in 

However, as will be shown in Chapter V, this is not the case 
in general although it may be true for R = 1/2 systematic 
fixed encoders. 

Example 2.1 Consider an R = 1/2 fixed polynomial encoder 
with 


G = 


^FREE 


11110000 .. 


111100 ... 


2 and r Q = r^ = . . . 


Clearly d Q = d 1 = — = d^ 


r^ = 4. Since ^ / r^, this en- 


coder does not have a feedforward inverse. | 

Consider an algebraic decoder which decodes in "frames" 
of + 1 blocks of received digits, t- » m, and which decodes 
each received sequence j^rj into the encoded sequence [x] 


l 


such that d 


( [ £ ] [y] )is minimum. A feedback decoder 


1 


operating over such a frame will make a decoding error for 

^FD + 1 

at least one pattern of — errors in the frame where- 

£ 

as the above algebraic decoder cannot make an error unless 


c l- 


■m 


2 


errors occur in the frame. 
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Now consider the binary symmetric channel (BSC) shown 
in Figure 2.1. p is the digit error probability. Let q be 
the probability that e digits were received incorrectly in n 
successive uses of the channel. Clearly 

q = p S (1 - p) n “ S . (67) 


Now let q' be the probability that e + 1 digits were received 
incorrectly in n successive uses of the channel. Then 


q' 


e+l,_ . n-e-1 qp 

■ p (1 - p> = T~^~p 


( 68 ) 


If p is very small, i.e., p « 1, then 

q' « q . 


(69) 


Therefore if p is small enough it can be seen that the de- 
coding error probability is a function only of the minimum 
number of incorrectly received digits in the frame that can 
cause a decoding error, since heavier error patterns occur 
with negligible probability. Since in general r^_ m > d^/ 
decoding error probability is lower for the decoder operating 
over the whole frame than for the feedback decoder. 

Therefore r j^_ m is the appropriate distance measure for 
the decoder which decodes in frames of -t + 1 received blocks 
since it determines the decoding error probability. Property 
F4 implies that if l is large enough, r^_ m = ^ FREE and dp^g 
becomes the appropriate distance measure for the decoder which 
decodes in frames of t + 1 received blocks. 

A sequential decoder, although not algebraic, does de- 
code in frames of many received blocks. Hence dp REE would 
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seem to be a more appropriate distance measure than either 
dp D or for encoders used with sequential decoding. Chap- 
ter VI presents simulation results which corroborate this 
rea soning. 

D. Reverse Distance 

Consider an encoder G with generator matrix 


G = 


G (0) G (0) . . . (3 (0) 
0 “1 ina 


fi ^j 1 ) -*- S m-l 


( 1 ) 


Q. . 


G U) 

** 


(70) 


Then for any of the distance measures defined in this chapter, 
the following definition holds. 

Definition 2.15 The reverse distance of the encoder G is 
the distance of the encoder G' with generator matrix 


G 1 


G (0) G (0)...g_(0> 0 .. 

0 G (1)...G (1) G '<!).. . | 

— m 1 0 1 


If G‘ = G, then the encoder G is called reversible . Rever- 
sible encoders have been studied by Massey [l4 ] and Robinson 



Reverse distance will prove useful in Chapter V, when 
some methods for calculating dp REE ate derived. 



III. Encoder Equivalence and Syndrome Formation 
A. Encoder Equivalence 

There is considerable interest in finding systematic 
equivalents for non- systematic encoders for the following 
reasons? (1) systematic encoders are in general simpler to 
implement than non- systematic encoders; (2) the encoded 
sequences from systematic encoders possess the "quick look" 
property, i.e., the noisy version of the information sequence 
is directly available at the receiver. 

As will be shown in later chapters, for a given rate 
and a given constraint length, non- systematic encoders are 
superior to systematic encoders for sequential decoding be- 
cause larger free distances are achievable. For some alge- 
braic decoding techniques, however, such as feedback de- 
coding, where column distance is the important parameter, 
non-sy sterna tic encoders are no longer superior and it is 
usually desirable to use only systematic encoders. 

1. Rational Equivalence 

Definition 3.1 Two encoders are rationally equivalent if 
they have the same set of output sequences over the set of 
all rational input sequences. | 

Consider a fixed non- systematic encoder with a matrix 
of generator functions G(D) . Then 

y(D> = x (D) G (D) (71) 

= x(D) R(D) R -1 ( D ) G(D) > (72) 
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where R~^(D) is the K x K inverse matrix of one of the non- 
singular K x K subnatrices of G(D) . (The definition of a con- 
volutional encoder guarantees that some K x K submatrix of 
G(D) is non-singular. ) 

Therefore 


y(D) = 2(D) 2(D) 


(73) 


where 2( D ) 


/\, 


x(D) R (D) , G(D) 


R -1 (D) G(D) , and 2(D) is in 


/\ 


systematic form. Hence if G(D) is realizable, it is a system- 
atic rational equivalent of G(D) . However, in general there 
is no guarantee that 2(D) be realizable. 

Since R(D) is just a K x K submatrix of G(D) , it contains 
only realizable functions. Therefore R - ^(D) , the matrix of 


cofactors of R(D) divided by det 


realizable functions if det 


R(D) 


R(D) , 


contains only 


has a non-zero constant 




term. Hence a sufficient condition for G(D) to be realizable 


is that det 


[r(D) 


have a non-zero constant term. 

Example 3.1 Consider the R = 2/3 fixed binary non-systematic 

fl 1+D d! fl 1+D 1 

encoder with G (D) =| D 1+D 2 1 j . Choose R (D) = D i + d 2 • 

Then det [r(D>] = 1 + D and E /(D) = [ “ +D " /1+D ] . 

J l! 


Hence 2(D) = 


1 

0 


0 

1 


1+D+D 

1+D 


is a systematic rational equi- 


valent of G (D) 

Example 3.2 Consider the R = 2/3 fixed binary non-systematic 
encoder with G(D) 

Then det 


■ 1 

1+D 

D ‘ 


' 1 

1+D' 

- D 

D 

D - 

. Choose R(D) = 

f- 

. D 

D . 


j^R (D)J = D 2 and R" 1 (D) = 


1/D 

1/D 


1+D/D' 

1/D 2 


Hence 
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' 1 

0 

1/D “ 

G(D) = 

. 0 

1 

1+D/D _ 

chooses 

R(D) 

= 

' 1+D 




_ D 


is not realizable. 


D 

D 


, then det 


[r(dj] 


But if one 
= D and R _1 (D) = 


"1 1 

_1 1+D/D J . 

In this case <GXD) 


1+D 1 0 

is in systematic form 

_ D 0 1 . 


and is realizable even though det ^R(D)J has a zero constant 
term. Hence this gXd) is a systematic rational equivalent of 


G(D).| 


In example 3.2, note that d^ = 2 for the encoder with 
while the information sequence [xj = 

[oi,loj produces a codeword [yj = [000,00l] with weight 1 

1 r 1 1+D 


G(D) = 


1+D 1 0 

D 0 1 


for the rationally equivalent encoder with G(D) 


LD D 


D 


Dj 


/ 


so that d^ = 1 for this code. Hence rational equivalence does 
not necessarily imply equivalence of column distances. 

2. Causal Equivalence 

Definition 3.2 The set of causally driven output sequences 
of the encoder G, (CD0S)_, is the set of all output sequences 
produced by causal input sequences. | 

Definition 3.3 Two encoders G and (j are causally equivalent 
if they have the same set of causally driven output sequences, 
i.e. , if (CD0 S) q = (CD0S}£. | 

Lemma 3.1 For two fixed encoders G and 'G', if (CD0S)_, = 

- r - ■’ {j 

( CDOS ) then for any two causal information sequences x and 

(jr 

jc such that x G = x G", x„^ 0 if and only if x_ ^ 0. 

“"“U u 



46 


Proof Assume x» = 0, xl ^ 0. Let n be the least positive 

— — — -*0 “ 0 — 

integer such that 0. Then the information sequence 

whose transform is D n x(D) is causal and produces the output 
sequence whose transform is y(D) = D” n x(D) G(D) . Therefore 
y(D) = D ^x(D) 'G'(D) . Since the definition of an encoder 
implies that no two information sequences can produce the 
same output sequence, D ^(D) is the transform of the only 
input sequence to G which can produce y. But the sequence 
whose transform is D n x(D) is not causal, contradicting the 
causal equivalence of G and G. The only if part of the 
proof follows exactly as above. | 

For a fixed encoder G, assume det £r(D)J has a non- 
zero constant term, i.e., R 1 (d) is realizable. Then if 
x(D) is causal, x(D) = x(D) R ^ (D) is causal ahd.it follows 
from equations (71) and (73) that 


(CD0S) £ C (CDQS) • (74) 

(j G 

Also if x (D) is causal, then x(D) = x(D) R(D) is causal and 
( CDOS ) Q C (CDOS)A . (75) 

Hence 


(CD0S) g = (CDOS) A 


(76) 


and the two encoders have the same set of causally driven 
output sequences. Therefore lemma 3.1 implies that 

A 


where 

order 


d. 

J 

j 


d j = d j' j = 0 , 1 , 2 ,..., (77) 

A 

is the order j column distance of G and dj is the 
column distance of Theorem 3.1 summarizes the 


above results. 
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Theorem 3,1 Let R (D) be a Kx K matrix formed with K columns 


of G(D) . If det 


R(D) 


has a non- zero constant term, then 


G (D) = R ^ (D) G(D) is a systematic causal equivalent of G(D) 


and d. = d . / j =0,1,2,... . | 
3 3 


If det 


R(D) 


(k) 


has a zero constant term for all 
possible choices of R(D) , a systematic causal equivalent may 
not exist. In this case, as will next be shown, a ration- 


ally equivalent encoder G can be found such that d < d , 

j j 

j = 0,1,2,..., and such that G has a systematic causal 
equivalent. The following procedure will produce the en- 

/V 

coder G. (Assume the numerators and denominators of the gen- 
erator functions of G are relatively prime polynomials.) 

Step 1 Convert G(D) to a polynomial matrix G' (D) by multi- 
plying each row of G(D) by the least common multiple of the 
denominators of its generator functions. | 

Clearly this step preserves rational equivalence. 


G' (D) = 


l i ( d) 


0 


l 2 ( d) 


0 


0 

0 


* L K (D) 


G(D) 


(78) 


where L^(D) is the least common multiple of the denominators 

of the generator functions in row i, 1 < i <1 K. Let 

0 


L(D) = 


L X (D) . 


0 


l k ( d) 


(79) 
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Then 


L (D) = 


' 1 /^( 0 ) 


(80) 


o ... i/l k (d) 

is realizable since G(D) realizable implies that each L i ( r) ) 
has a non-zero constant term. Therefore if x 1 (D) is causal. 


then x(D) = x' (D) L(D) 


-[ 


x^ (D) (D) , . . . , x (K) 


(D) L. 


K 
-1 


(D)] 


is causal. Also if x(D) is causal, then x' (D) = x( D ) L (D) 
= [x (1) (D) /Lj (D) , . . . ,x (K) (D)/L k (D)] is causal. Hence step 1 
preserves causal equivalence. 

Note that since det j^R(D)J is assumed to have a zero 
constant term, det j^R Q ^ = 0 for all | ^ j possible choices of 
r(D) , where Rq is the K x K matrix of constant terms of 


R (D) . Therefore rank [fio] 

M 


< K. 


< K and since G ' = I T .G_, rank 
— 0 KU 


th 


Step 2 Convert G 1 (D) to a matrix G" (D) in which the K 

tTl, 

row of G", i.e., the constant terms in the K row of G" (D) , 
—0 “ 

is all-zero by rearranging the rows of G ' (D) and then adding 
a linear scalar combination of the first (K-l) rows of G ' (D) 
to row K. | 

Again this step clearly preserves rational equivalence. 

After rearranging the rows of G ' (D) , 

G' (D) , (81) 


G" (D) 


1 0 
0 1 


0 

0 


C 1 C 2 


K 
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Where each C. e GF(2) and C jt 0. Let 
1 K 



Clearly C(D) is non-singular and the proof that causal 
equivalence is preserved is similar to the proof in step 1. 
Step 3 Convert G" (D) to a matrix G" ' (D) by multiplying 
row K of G" (D) by D _1 . | 

Since the row space of G" (D) over all rational input sequences 

tin ““X 

is not changed by multiplying the K row of G" (D) by D , 

G" 1 (D) is rationally equivalent to G" (D) . 


G" 1 (D) = E(D) G" (D) , 


where 

E(D) 


1 0 
0 1 


0 

0 


0 



(83) 


(84) 


is non-singular and non-real izable. Note that 


E _1 (D) 


1 0 
0 1 


0 

0 


(85) 


|_0 0 ... DJ 

is realizable. Therefore if x" (D) is causal, then x n ' (D) = 
-I 

x" (D) E (D) is causal and 


( CDOS ) Q , , C (CDOS) g „, 


( 86 ) 
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However, the encoders G" and G" 1 are not in general 

causally equivalent, since x" 1 (D) causal does not imply that 

x" (D) = x" 1 (D) E (D) is causal. Therefore the (CDOS) set is 

in general enlarged in step 3, and the encoder G" 1 is said 

to be causally dominant to the encoder G" . It remains to 

show that d^! 1 £ d|j , j = 0,1,2,..., where d .'l ' is the order j 

column distance of G" 1 and d'! is the order j column distance 

J 

of G". 

Case 1 Let x" 1 (D) be the transform of a causal input se- 
quence with xJJ'^ 0, x q v = 0. Then x" (D) = x" ' (D) E(D) 

is causal and has x"q ^ 0. Hence y" (D) = x" (D) G7*(D) = x" 1 (D) 

G" ' (D) = y" 1 (D) and w H ( [y"j ) =w R t ['£"•]), j =0,1,2,... ,| 

Case 2 Let x" 1 (D) be the transform of a causal input se- 
quence with x 7 ^ 0 and let y" 1 (D) = x" ' (D) G" ' (D) . Then 

x" (D) = Dx" (D) = Dx" ' (D) E (D) = [ Dx* 1 * " ' (D) , . . . , Dx* K-1 * " ' (D) , 

x (D) J is causal, has x^ ^ 0, and produces an output se- 

quence whose transform is y" (D) = x" (D) G" (D) = x" (D) E~^(D) 

G" ' (D) = Dy" ' (D) . Clearly w H ( [y"] ) ^ w H ([y"'] )/ J = 0,1,2, 

I 2 2 

« « » « | 

Cases 1 and 2 imply that d 1 ! ' > d'i , j = 0,1,2,... . Therefore 

J 3 

causal dominance implies that the column distances cannot be 
decreased. Also, equations (83) and (84) imply that m" 1 < m", 
where m" 1 is the memory of G" 1 and m" is the memory of G" . 

At this point, steps 2 and 3 are repeated until an en- 
coder G is obtained such that rankj^G Q J= K, m < m, and G 

is causally dominant to G. Then a K x K submatrix ]|(D) of 
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G(D) can be found such that det R (D)J has a non-zero con- 
stant term. Hence from theorem 3.1 a systematic encoder G 
with G (D) = R (D) G(D) can be found which is rationally 
equivalent to and causally dominant to G. These results are 
summarized in the following definition and theorem. 

Definition 3.5 The encoder G" ' is causally dominant to 
the encoder G" if (CDOS) G „, D (CDOS) Gll .| 

Theorem 3.2 For an encoder G with a matrix of generator 
functions G(D) , if no K x K sufcmatrix R(D) of G(D) exists 
such that det ^R(D)J has a non-zero constant tern, then a 
systematic encoder ^ can be found which is rationally equiva- 
lent to and causally dominant to G. | 

/V 

If the above procedure never results in an encoder G 
such that a K x K submatrix R(D) of G (D) can be found with 
det ^ — o J ^ then can ke sh own that the procedure pro- 
duces a matrix with an all-zero row after at most Km 1 +1 
applications of step 3, where m’ is the memory of the poly- 
nomial encoder G 1 . This follows because each time step 3 is 
applied, the memory of one of the K generators must be re- 
duced. In this case the original encoder G has no inverse 
and dj =0, j =0,1,2,... . 


Example 3.3 Consider the R = 2/3 fixed binary non -sy sterna - 

r 2 ~' 

tic encoder G with G (D) = 1 1 1+0 1+D 


So = 


111 

111 


1+D 1+D+D 1 

and rank |j2qJ = 1- After adding row 1 to 


-1 


row 2 and multiplying row 2 by D , the encoder G with 
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G(D) 


1 



R(D) 



Hence 


6(D) 


1+D 1+D 2 

D D 


results. Choosing 


1+D 

D 

~—x 

= R (D) 



= 1 and R -1 (D) = 

0 d 2 +d 3 

2 

1 1+D+D 


D 1+D 

1 1 

is the matrix 


of generator functions for the systematic encoder G which is 
rationally equivalent to and causally dominant to G.| 

It should be noted here that Forney's [6] method of pro- 
ducing a "canonic" non-systematic encoder does not consider 
the problem of preserving column distance. 

The results of Section III. A can be summarized by stat- 
ing that there is no loss of generality in considering only 
systematic encoders for algebraic feedback decoding tech- 
niques where column distance is the important parameter. How- 
ever, in many cases a non-systematic polynomial encoder G' 
with encoding memory m' may have a causally dominant, sys- 
tematic, rationally equivalent encoder G with infinite en- 
coding memory, i.e., G(D) may contain rational functions. 

Since it is often undesirable to have feedback in the encoder, 
^ can be converted to a systematic polynomial encoder of 
approximately the same complexity as the encoder G' by trun- 
cating each generator function in G(D) after degree m' . 

Clearly the encoding memory of Cj' is m' and dj = d^ > dj , 
j = 0,1,..., m’. But the encoder G' is no longer rationally 
equivalent to G 1 , and hence may have a lower value of <ip REE « 



Therefore the code produced by the systematic encoder G' 
may not perform as well with sequential decoding as the 
code produced by the non- systematic encoder G' with the 
same encoding memory. 

B . Syndrome Formation 

The parity-check matrix for all systematic encoders 
was given in Chapter I. In this section parity-check 
matrices and syndrome forming circuits will be given for 
all fixed non -systematic polynomial encoders G such that 
rank ^GqJ = K. These are then valid parity-check matrices 
and syndrome forming circuits for all rationally equivalent 
encoders (cf. equations (30) through (36)) . That every 
fixed non- systematic encoder is rationally equivalent to a 
fixed non- systematic polynomial encoder G such that rank 
[^gJ = K was shown in the previous section. 

The parity-check matrix for R = ~ fixed non-systematic 
encoders is given by 


H T (D) = 


The syndrome forming circuit for these encoders is shown in 
Figure 3.1. 


g 1 (2) (D) 
G, (1) (D) 


G 1 (3) (D) 


G 1 (1) (D) 


.. g/^d) 
0 

0 


(87) 


0 ... G 1 (1) (D) 




Pig. 3.1. Syndrome forming c 



1 

for R = n encoders. 
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T 

However for non-systematic encoders with K > 2, H (D) 
contains products of generator functions. Consider for ex- 
ample an R = 2/3 fixed non-systematic encoder with 

(D) 


G(D) = 


U) (D) 

g/ 2 * CD) 

G (3) 
G 1 

(1> (D) 

G 2 (2, (D) 

Q 

to 

(jO 


(D> 


( 88 ) 


and 


H T (D) = 


H (1 > (D) 
H< 2 > (D> 
H (3 > CD) 


Since G (D) H 1 (D) = 0, 


G^ 1 * (D) G x (2) (D) G-^^D) 


( 1 ) 


( 2 ) 


(3) 


G„'-'(D) G 0 ' ; (D) G'~'(D) 


«l 

H^ 1 ^ (D) 


0 


H (2) (D) 

2S 



_h (3) (D) _ 


0 


(89) 


(90) 


Therefore 

G 1 (1) (D) H (1) (D) +G X (2J (D) i{ 2 (d) + G x (3) (D) H (3) (D) =0, 

(91) 

g 2 {1> (d) h (1) (d) + g 2 (2> (d) h (2J (d) +g 2 (3) (d) h (3) (d) = 0, 

(92) 


and 

H (1) (D) = 


G 2 (2) (D) H (2) (D) + (D) H (3 > (D) 

G 1 (1) (D) 


(93) 


Hence 

G 2 (1 ^(D) Gj ^ tD) H (2) (D) +G 2 (1 ^(D) G 1 (3) (D) H (3) (D) 

G 1 (1) (D) 

+ G^ 2 * (D) H* 2 * (D) + G 2 (3) (D) H * 3 * (D) = 0 


(94) 


and 



H (2 >(D> = 



G (1) (D)G 1 (3) (D) 

h (3) (d) 

f3Y 2 1 

G 2 <d > + 3^75 


g 2 (2) (d) + 


^21 (D) 
G-^ 1 * (D) 
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(95) 


Now choose 

H (3) (D) = G-j^ ( (D)G 2 (2) (D) + G (1) (D)G 1 (2) (D) . 


(96) 


Then 

( 2 ) 


( 1 ) 


(3) 


( 1 ) 


(3) 


H^' (D) = G 1 ^ / (D) G’ 2 v ' (D) + G 2 ' - (D)G 1 W/ (D) . 

Finally 

G 


(97) 


H (1) (D) 


\ {2} (D) 


G x ( 1) (D) G 2 ( 3> (D) + G 2 ( 1) (D) G 




+ G X (3 ) (D) 


G 1 (1) (D) 


(98) 


G 1 (1> (D)G 2 (2> (D) + G 2 (1> (D)G 1 (2) (D) 


(1) , v 

G 1 (D> 

= g 1 (2) (d) g 2 (d) + g 1 (3) (d>g 2 (2) (D) . 
Therefore if m is chosen according to equation (53) , 


(99) 


m = max [n 11 “ 22 ^m 21 m 12 ,m 11 m 23/ m 21 m 1 3 / m 12 m 2 3 f m 1 ^n 22 J , (100) 


where nu ^ = degree 


G ± (:i) (D) 


for all i and j/and the de- 
coding memory, or syndrome circuit memory, is on the order 
of twice the encoding memory. 

The syndrome forming circuit for R = 2/3 fixed non- 


systematic encoders is shown in Figure 3.2. 
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Pig. 3.2. Syndrome forming circuit for R = j encoders. 
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Example 3 . 4 Consider the encoder described by 
1 1+D 1+D+D 2 

G(D) = 

D 2 1+D+D 3 1 

Then 

4 5 

D+D+d 

2 3 4 

1+D+D +D 

1+D+D 2 

In this case the encoding memory m = 3 and, if chosen ac- 
cording to equation (53) , the syndrome circuit memory m = 5. 
The syndrome equations from time u through time u+5 are then 





given by 



1 00 1 0 0 II 1 0 1 

I 1 00 1 0 01 I 101 

I 100 10 01 I 101 

I 1001 0 01 1101 

110010 0 I I 1 0 1 

I 1 00 1 0 Oil 101 


000 1 1 1 
000 1 I I 
000 1 I I 
0001 I I 


000 1 I I 
0001 I 
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By generalizing from the R = 2/3 case, the form of the 


parity-check matrix for all fixed non- systematic encoders 
can readily be obtained. Let R (D) be the first K columns of 
G(D) „ Assume rank [ Sq J = Then denote det j^R(D}J by A 
and the cofactor of ^ (D) in R(D) by A^j . Then 




il G i 


(K+2> 


(D) ... £ A i:L G i (N> (D) 




(K+2) 


i 


K * 


'«•>-£ ^ 1 <K+1, <» - I ^ 


( 102 ) 


It can easily be verified that equation (102) reduces to the 
familiar form of H (D) for R = 2/3 and R = 1/N fixed non- 
systematic encoders and for all fixed systematic encoders. 

Note that for fixed systematic encoders A = 1. The syndrome 
forming circuits for R = 2/N and R = 3/N fixed non- systema- 
tic encoders are given in Figures 3.3 and 3.4, respectively. 
Equation (102) has also been implicitly derived by Forney [6] . 
For a fixed non- systematic encoder G such that rank 

Gq = k, the code produced by G is exactly the same as the 

T 

null space of H . However, as discussed in Chapter I, if 



encoders. 
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rank |^2 qJ < K, the null space of H T contains other sequences 

besides the set of output sequences of G. In some instances, 

as in Chapter VII, it is convenient to define a code with a 

parity-check matrix. In order to avoid ambiguity, the code 

defined by a parity™ che ck matrix will be taken to be pre- 

T 

cisely the null space of H . 

In keeping with standard usage, throughout the re- 
mainder of this thesis many of the properties precisely de- 
fined for encoders will be referred to as properties of the 
code produced by a given encoder. For example, an R = 1/N 
fixed systematic code is the code produced by an R = 1/N 
fixed systematic encoder. 



IV. Bounds on Distance 


A. Introduction 

A complete set of bounds on the distance properties of 
convolutional codes does not yet exist. However, many re- 
sults in this direction have been obtained. Wozencraft and 
Reiffen [ 3 ] proved a Gilbert lower bound on dp D for R = j~ 
binary fixed codes which Massey [ 4 ] later generalized to all 
rates and to GF(q). Robinson [ 16 ] proved an upper bound on 
dpj^ for fixed codes that is asymptotically a Plotkin bound, 
and Massey [8] later gave a simple bound that has the same 
asymptotic form. 

Robinson [ 7 ] also obtained a lower bound on d^ for 
systematic fixed codes, in which d^ grows only as the square 
root of n^. Kolor [ 17 ] , for R = 1/2 systematic binary 
fixed codes only, and then Massey [8] for all rates, proved 
a lower bound on d^ in which d^ D grows linearly with . 

Wagner [ 18 ] obtained a lower bound on d^ for non system- 
atic periodic codes with R < 1/2. A new lower bound good for 
all rates will be given in this chapter. Wagner's results 
will also be extended to obtain a Gilbert lower bound on d^, 
for a subclass of periodic codes with period T = 2m + 1 which 
does not include fixed codes as a special case. Since fixed 
codes are a special case of the entire class of periodic codes, 
the usual Gilbert lower bound on dp D holds over the whole en- 
semble of periodic codes. 
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No upper bounds on are known except those that hold 

trivially since <1 ^ < d 

DD FD 

Since dp D < d^^, the usual Gilbert lower bound on 
dp D is also a bound on dp REE , albeit a weak one. Neumann [ll] 
proved a much stronger lower bound on dp REE for fixed non- 
systematic codes. In this chapter a still stronger lower 
bound on dp REE will be given for non -systematic periodic codes 
and this result will be used to obtain an improved upper bound 
on error probability for non-sy sterna tic periodic codes used 
over the BSC and with a maximum likelihood decoding rule. 

McEliece and Rumsey [lo] obtained a Plotkin upper bound 
for R = — systematic fixed codes. The extension of 
this result to all rates, to non- systematic fixed codes, and 
to periodic codes will be given in this chapter. 

Finally a Gilbert lower bound on d^ for an important 
subclass of R = 1/2 non- systematic fixed codes will be ob- 
tained. This bound has application in the chapter on code con- 
struction. 

B. Bounds on dp D 

1 . Lower Bounds 

A lower bound on distance guarantees that at least one 
code can be found with distance greater than or equal to the 
lower bound. Wozencraft and Reiffen [ 3 ] and Massey [ 4 ] have 
shown that there exists at least one binary fixed code such 
that 

^FD 

lim - — > H" 1 (l - R) , 
m co fd 



( 103 ) 
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where H(x) = -x log^ x “ (1 - x) 10^2 ^ “ x ) is the binary 
entropy function* Equation (103) is called a Gilbert lower 
bound because it is asymptotically the same as Gilbert's lower 
bound on the minimum distance of a block code [l9] . 

2. Upper Bounds 

An upper bound on distance guarantees that no code can 
be found with distance greater than the upper bound. Robinson 
[ 16 ] and Massey [8] have shown that for all binary fixed codes 


^FD 

lira “ - < ~ (1 - R) . (104) 

m - oo n FD 

Equation (104) is called a Plotkin upper bound because it is 
asymptotically the same as Plotkin ' s upper bound on the mini- 
mum distance of a block code [ 20 j . An upper bound which is 
asymptotically the same as Hamming's [21 j upper bound on the 
minimum distance of a block code is called a Hamming upper 
bound and an upper bound which is asymptotically the same as 
Elias's [ 22 j upper bound on the minimum distance of a block 
code is called an Elias upper bound. However no Hamming or 
Elias upper bounds are yet known on d^ for either periodic 
or fixed codes. 

C. Bounds on d^ 

1. Fixed Codes 

Generalizing the work of Kolor [ 17 ] , Massey [ 8 ] has 
shown that there exists at least one binary systematic code 
such that 


lim 

m -*■ °° 


*^DD 


n 


DD 


aH ( JL 


1-R 
1+R * 


(105) 
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Note that this bound guarantees a linear increase of d^ with 
n^. In Robinson's [ 7 ] earlier bound, d^ is guaranteed to 
increase only as the square root of n^» 

The Plotkin upper bound on d^ of equation (104) also 
holds for d^ since d^ < d^, but no other upper bounds on 
d [)D are known for fixed codes* 

2. Periodic Codes 


For periodic codes, Wagner [ 18 ] has shown that there ex- 
ists at least one code such that 



__lim 
m -* 00 

d w 

n~ * H 
w 

" 1 (1 - 2R) 

/ 

(106) 

where 






d 

w 

= min 
m<u<m+T 

min 

x /x 1 

— U “ u 

a H ( ( 2 

S 1 

_ / [x'g] J = 

u,u+m u,u+m 

min 

in<u<m+T 

min 

x ,/x ' 
—u u 

VM 

- ' | 
u , u+m 


„ nJ -> and n w = n PD 
u,u+m 


are just different definitions of definite decoding minimum 
distance and definite decoding constraint length than those 
given in Chapter II* Note that this bound is only good for 
R < 1/2. A bound which is good for all rates can easily be 
derived using Robinson's and Massey's more natural defini- 
tions of the definite decoding parameters. First consider 
only canonic systematic codes with m = m. 

For a canonic systematic code with period T = m + 1, 
the equations for the parity sequence from time 2m through 
time 3m can be written as 


2 . 


2m 


x Q 
2m —0 


(m - 1) + x 


2m-l 


Q-, (m - 1) + 


+ x Q (m 

—m ifm 


1 ) 
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fi 2m + l = * 2 m + l 2o (m) + * 2ra Si« + ••• + 4n + l ^ (m) 


(107) 


film " —3m 9o <m ' 2) + — 3m-l 2i<“- 2 > + - • - + em (m - 2, ' 

where each Q^(u) , 0 < i, u < m, is defined as in equations 
(16) and (17). 

Theorem 4.1* There exists at least one systematic periodic 
time-varying convolutional code such that 


m -*■ oo no 


MHf) 


In particular, there exists at least one periodic oode with 


period T = m + 1 such that 

d DD 

lim n — > 

m -»• oo DD 


•1 / 1 - R \ 
[TTr ) 


Proof For a given definite decoding codeword x , ...,x„ _ , 

~m 2m- 1 


, X _ - , X _ , P , . . . ,x , P- 

2m-l 2m 2m 3m 3m 


, there 


are (m + 1) K (N - K) unknowns in equation (107) . For X 2 m ^ 

0, equations (107) are linearly independent. Therefore since 
there are (m + 1) (N - K) equations, each codeword with ^ 
2 (m+D 2 K (N-K) 

0 has y = * (m+1) (N-K) solutions. Finally, since T = m + 1, 

each codeword with some ^ 0, 2m < u < 2m + T, belongs to 
at most (m + 1) y different codes. 

The number of codewords with Hamming weight less than 


or equal to d is 


< 2 n DD H \ n DD 


when r* — <1/2 

"DD 


This result was obtained independently by Morrissey 23 
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Hence the number of codewords with X 2 m ^ — an< ^ weight less 

than or equal to d is less than 2*^20 H (n DD ) . Therefore if 

(m + 1^2^00 1-1 ( n^ ) is less than the total number of codes, 
there exists at least one code with d^ > d. Equivalently, 


H 


if d^ is the smallest integer such that (m + l)y2 n PP 
2 

;> K(N-K) , then there exists at least one code with 


(fES) 
Wp / 


definite decoding minimum distance ^DD* But 

d 

l 1 

mK + (m + 1)N 


(m + l)y2 


H 


DP 

.*5d 


> 2 


(m+l> K(N-K) 


log 0 (m + 1) + (m + 1) * k(N-K) - (m + 1) (N - K) + 


(m + 1) N 


H 


^DD \ 


mK + 


n^ ) > (m + 1) K(N - K) 

mK + (m + 1) N 


log^ (m + 1) - (m + 1) (N - K) + 


H 1 


mK + (m + 1)N 


H 


\>d\ 1 °g 2 (m+1) 


n DD y + mK+(m+l) N " rnK+(m+l)N 


(m+1) (N-K) 


DP 

»PP 


^ 0 


> 0 


H 


H 


d pp\ (N-K) (m+1) log 2 (m+1) 


“pp, 


^ mK+ (m+l) N “ mK+(m+l)N 


Wd/ 


lim 

m -* oo 


. 1 - R 

> asm-* 00 

d 

PP -i 

=55 SH 


( i - R \ 

\ 1 + R / • 


This bound is plotted and compared with the usual Gilbert 
lower bound on d^ in Figure 4.1. 


Massey [8] has conjectured that his bound on d^ for 
systematic fixed codes should be the same as the bound in 
theorem 4.1. He claims that the factor of in equation (105) 
should be eliminated by tighter arguments. 
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For non- systematic codes with decoding memory m = m 
and period T = 2m + 1, the equations for the transmitted se- 
quence from time m through time 3m can be written as 

y = x G (m) + x .G, (m) + . . . + x~G (m) 

■%t -m— 0 -m-l—l — 0~m 


= x_ G (2m) + x G. (2m) + 
JL 2m —2m 0 2m -1—1 


+ x G (2m) 
“tn-m 



a 3rrft) (m - 1) + S 3m-l S l <n, - 1) + •" + ^mV 1 "- 1 * * 


Theorem 4.2 There exists at least one non-systematic per- 
iodic time-varying convolutional code such that 

doD r 

lim ^ — > min H ^ (1 - , 1/2 H ^(1 - R) 

m QQ DD L J 

In particular, there exists at least one periodic code with 


period T = 2m + 1 such that 

^DD [ -i 

lim ~ — > min H (1 

m -*■ oo 1l dd 



Proof For a given definite decoding codeword Y. 2 m ' 

2 

. . . , Z 3 m / there are (m + 1) NK unknowns and (2m + 1) N 
equations in equations (108) . But X 2 m / Sl guarantees only 
that the last (m + 1)N equations are linearly independent. 
Information blocks Xq through X 2 m ,„]_ could be all -zero. 

Assume Xj is the first non-zero information block, 
j = 0,1,..., 2m. If j > m, then the usual Gilbert lower bound 
on dp D holds for the definite decoding minimum distance, i.e.. 



71 


oo TFD 


DD -1 DD , „i 

n — > H (1 - R) or lira 1/2 H (1 - R) since 


oo DD 


m = m implies that n =(2m+l)N**»2(m + l)N = 2n pD for 
large m. However, if j < in, all codewords produced by an 
information sequence containing a span of m consecutive all- 
zero information blocks inclusive between block j and block 
(2m - 1) have Hamming weight at least as great as the mini- 


mum weight codeword with x Q = 


= — 2m-l i ' e " at 


least as great as d^. Hence all such codewords need not be 
considered since they cannot possibly be the minimum weight 
codeword with x 2m / 0. Therefore all of equations (108) are 

independent, and for each codeword with x 2m ^ 0 there are 

NK (m+1) ( 2m +1) 


iNTzm-i 


solutions to equations (108) . 

Finally, since T = 2m + 1 and the information digits can 


be chosen in any of 


K , 
2 “1 


0 (3m+l)K , , , 

2 different ways, each code- 


word with some x / 0, 2m < u < 2m+T, belongs to at most 
— u 

K . 

,v 2 ~ I _ ( 3m+l) K . . _ ( 3m+l) K 

( 2m + 1) 2 (2m + 1) 2 Y different codes. 

Therefore, proceeding as in the proof of theorem 4.1, 
if d__ is the smallest integer such that (2m + 1) 2 (3m+l)K y 


2”dd h ) i 2 NK <" +1 > < 2ra+1 >, then there 


exists at least 


one code with definite decoding minimum distance cL.^. But 
/«D D\ 


(2m + 1) 2 <3m+1,K y 2 < 2m+l) N H 


W * 2 


MK (m+1) ( 2m+l) 


log 2 (2m+l) + ( 3m+l) K - N (2m+l) + (2m+l)N H 
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(2m + 1) N 

.w 


DD 


H 


n 


- 1 + 


DD) 


( 3m+l) K 
( 2m+l) N 


H 


]- 1 + 3/2 R > 0 as m 00 
1 


m 


^DD 
oo ^DD 


lim ^ — £ H (1 - 3/2 R) 


+ 


log 2 ( 2m+l) 
( 2m+l) N 


Therefore 


^DD 

lim ~ — > min 



3/2 R) , 1/2 h" 1 (1 - R) . 1 


m — co DD 

Note that this bound holds only for R < 2/3. 

Theorem 4.2 can be extended to all rates by redefining 
dpp and n^p. Assume = (3m + 1)N and T = 3m + 1. Then 

d_ D = min min w ( [ y; 1 ) , and after a 

3m<u<3m+T x /0 u-2m,u+m 

— u — 

slight modification of the proof of theorem 4.2, the following 
corollary results. 


Corollary 4.1 There exists at least one non- systematic 


periodic time-varying convolutional code such that 


lim 
m -*■ °° 


dpp 

^D 


> min 


H" 1 (1 - 4/3 R) , 1/3 H _1 (1 - R) 


In general, let = (Xm + 1)N and T = Xm + 1, X a positive 

> , 


integer. Then cL = min min w (fy; 

Xm<u<Xm+T x /O 
— u 


u+m-Xm,u4m 


and there exists at least one non- systematic periodic time- 

varying convolutional code such that lim > min h 

m -► oo ^dd 

. Note that this reduces to 


(1 


X+l 

X 


R) , J H _1 (1-R) 


Wagner's bound when X = 1, to theorem 4.2 when X. = 2, and to 
corollary 4.1 when X = 3. 
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Note also that the above bound holds for any T > Xm+1 

such that T is some algebraic function of m, i.e. , such that 

log 2 T 

T grows less than exponentially with m, since — still 

N (Xm+1) 

approaches 0 as m approaches 00 . It is well known [ 24 ] that 

bounds which use a fraction of codes argument in the proof, 
such as the above bounds, hold for almost all codes. Hence 
letting X R be the value of X which maximizes the above bound 
for a given R, the following corollary results. 

Corollary 4.2 For almost all non- systematic periodic time- 
varying convolutional codes with period T i> X R m+l such that 
T grows less than exponentially with m. 


dpD [ -1 

lim n ;> min H (1 - 

m -*■ 00 jJD 

Corollary 4.2 guarantees a linear growth of d^ with n DD for 
almost all non- systematic periodic codes, even though the 
bound is very weak for high rates. The two functions which 
comprise the bound of theorem 4.2 are plotted together with 
the usual Gilbert lower bound on dp D in Figure 4.2. 

Wagner also proved that the result of equation (106) 

holds for an easily instrumented class of systematic codes 

3m*4~l 

suggested by Massey with period T = — ~ if m is °<3d or 3m + 1 
if m is even. The R = 1/2 encoder in this class is shown in 
Figure 4.3. Once each second the top shift register shifts 
once while the bottom one shifts twice. It can also be shown 
that there exists at least one of these codes such that 


X n + 1 

R 1-1 
X D r), r h (1 - r) . | 

R. R 


n. 


DP 

DD 


> H -i 


lim 

m ^00 
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Pig. 4.2. A lower bound on d^ D for non- systematic periodic 
codes.. 


■Did 


75 



Pig. 4.3. An R » | periodic encoder with T = if 

m is odd or T * 3m+l if m is even. 


X2 -^^€PrD CP 



^0 y 2m 


m 


+' 


Pig. 4.4. An R = - periodic encoder with T = 2m+l . 
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Now a similar class of systematic codes will be pre- 
sented for which a Gilbert lower bound on d^ is proved. 

These codes do not include fixed codes as a special case and 

hence such a bound is interesting. These codes have period 
T = 2m + 1 and the R = 1/2 encoding circuit is shown in 
Figure 4.4. Once each second the top shift register shifts 
once while the bottom one shifts twice. This leads to the 
following equations for the parity vectors? 

(0) Eq = 2o £ 0 

U> % = % °2 + *0 Hi 

(2) E 2 = S 2 -A + — 1 H3 * — 0 H2 
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chosen arbitrarily, along with all the rows of Q 5m+2' 

s t 

. . . , Q 2m * In the (m - 1} equation, then, only Q m _ 1 has not 

s t 

been specified. The (m - 1) equation fixes one row of Q Jn _2 

and the others can be chosen arbitrarily. Similarly in each 

of the remaining m - 1 equations, only one row of one matrix 

need be fixed. Therefore a total of (m+1) (N-K) digits are 

fixed by the first m + 1 of equations (109) . Since there are 

2 ( 2m+l) K (N-K) 

(2m+l) K (N-K) unknowns, there are solu- 

2 

tions. | 

Theorem 4 . 3 There exists at least one systematic code of 
the type shown in Figure 4.4 and described by equations (109) 
such that 


lim 
m 00 


^FD 


* s'h 1 


R) 


Proof Since T = 2m + 1, each codeword y 


with some 


u,u+m 


x ^ 0, 0 < u < 2m + 1, can belong to at most (2m + 1) v 
_ u _ 

different codes. Therefore if dp D is the smallest integer 

' d T 


such that (2m + 1) y 2 n FD H 


FD 

^D 


( 2m+l) K (N-K) _ 

> 2 , then 


there exists at least one code with feedback decoding minimum 

distance dp D « But , ^ \ 

^ 1V ( 2m+l) K (N-K) - (m+1) (N~K) + (m+1) NH I “ ) . _ (2m+l)K(N-K) 

( 2m + 1) 2 \ “fD / ^ 2 


/ FD 

log 2 (2m + 1) + (m + 1)NH ^ 


(m + 1) (N-K) ^ 0 


(m + 1)N 

^FD \ 
n FD / 


d FI 


H 


lim 
m -► 00 


H ' n FD / “‘ 1 + N (m+1) + 
I > 1-R asm-*- 00 

1 


K (m+1) log 2 (2m+1 > 


N (m+1) 


> 0 


FD 

^D 


> H 


(1-R) 
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Theorem 4.3 guarantees that simply instrumented codes of the 
type shown in Figure 4.4 and described by equations (109) can 
be constructed with large <ip D » 

D. Bounds on d pREE 
1 . Lower Bounds 

a. Fixed Codes 

Clearly all lower bounds on d^ are also lower bounds on 
^REE since - ^REE* The onlY other lower bound on dp REE 


was given recently by Neumann [ 11 ] for non- systematic codes 
only. His result states that there exists at least one binary 


non- systematic fixed code such that 

.-I 


2H 


(1-R) 


FREE 

~nZ 


> 


2R (1-2 2R “ 1 ) 
H(1-2 2R_1 ) +2R-1 


for R > 0.37 . 


for R < 0.37. 


( 110 ) 


b. Periodic Codes 

A stronger bound than (110) can be obtained for binary 
non-systematic periodic codes. First a bound similar to pro- 
perty F4 must be proved for non-systematic periodic codes. 

Lemma 4.2 (L = r , for almost all non- 

[ N (m+1) -1 ] [ (X R +1) m] 

systematic periodic codes with period T > X m+1 such that T 

JR 

grows less than exponentially with m, where X is defined as 
in corollary 4.2. 

Proof Corollary 4. 2 guarantees a linear growth of with 
n^ and hence d > 0. This implies that dp D > 0 and for all 
information sequences x with Xq ^ Q.> the first m+1 blocks of 
transmitted digits must contain at least one 1. Assume n^ 
and d^j^ are defined as in corollary 4.2. As in the proof of 
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property F4, no information sequences with m or more consec- 
utive all-zero blocks need be considered. Hence d^ > 0 im- 
plies that the next (1+1) m blocks of transmitted digits must 
also contain at least one 1. Therefore since <3p REE £ N(m+1) 
and all the possible minimum free weight codewords must have 
weight at least N(m+1) after m+l+ ^N(m+1)-1 
mitted blocks, = r [ (Xr+1) „| • I 

Let ^ max - [N(m+1)-1 j | (k R +l)mj +1, the bound on the 

length (in blocks of K digits each) of information sequence 

needed to produce the minimum free weight codeword. Consider 

the ensemble of non-sy sterna tic periodic time-varying codes 

with T = Z + m. Clearly T is only an algebraic function 
max 

of m, i.e., T grows less than exponentially with m. Let 
be the set of all information sequences of length Z such that 

^ 2. 7 s 0, Xo = 2! = ... = ^ = 

... =0, for some u, 0 < u < T, and which contain no string 
of m or more all-zero blocks inclusive between block u and 
block u+-i-l. Then let F^t/d) be the fraction of codes with a 
codeword of weight d or less produced by an information se- 





Once one row of G has been specified as having low weight. 


the digits of the remaining 


distinct rows can 


pc-l + (T-l)K 

be chosen arbitrarily. Hence a low weight codeword produced 
by an information sequence from can appear in at most 

2 [k- 1+ (T-l) k] N (m+1) £ / N(m+1, j = 2 NKT(m+l)-N(m+l)^ / N(m+1) 

j=o ' J j=o ' J 

K 

codes. Since there are T(2 -l) ways of choosing such an in- 

d 


formation sequence, F(l,d) is at most 

<3. 

, , v r> / N (m+1) \ 

T(2 K 1) 2 ^KT (m+i) -N (m+1) \ l . 1 

j=0 


r. / N (m+1) \ „ / N (m+1) \ 

L \ i ) t(2 k -i )l ( j ) 

1=0 j=o 


_NKT (m+1) 


_N (m+1) 


N (m+i) 


i r, 

i=o 

(N -K) -i _ 2K+Nm 


for i = 2, 3, . . . ,i, 


max 


Lemma 4.4 T ( ^ 

F (i, d) ^ 

2 ' 2 ' 

Proof For a particular information sequence of length -t be- 
longing to S the transmitted codeword has a length of m+i 
blocks. ^ 

Hence there are 

j=0 

possible candidates for low weight codewords. The encoding 


N (m+i) \ 


low weight sequences that are 


equations for u=0 can be written as follows: 
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Xm ~ X* 2 0 (m) + *<*-1 Si lm) + • 


• + x n G (m) 
0 m 


(in) 


^l-l = s l- 1 ^ 0 ( ^" 1)! + -1-2 + ••• + %-m-l 


W-l = *1-1 ^n (m+ ^" 1> * 

Since no information sequences with m or more consecutive all- 
zero blocks are being considered, and T > m + 1, equations 
(111) , when put into matrix form, have rank equal to the number 
of equations. Therefore, given a particular information se- 
quence from S ^ and letting m+ ^_2 be a particular low weight 

2 TNK (m+1) 

sequence, there are — n ( m+1) — solutions to equations (111). 

^ Kd — 2) K 2 

Since there are at most 2 ' (2 -1) different information 


sequences in Sj, , 

T ( 2 K _i) 2 2 TNK(m+l) £ 

F(l,d) < 


d 

A 


N (m+1) \ 
j / 


2 N (m+i) 2 TNK (m+1) 
T(2 K -1) 2 £ / N<m+t! \ 


< 


j=0 




2 1(N-K) 2 2K+Nm 


Theorem 4.4 There exists at least one non-systematic per- 
iodic code such that 


lim 
m -*• °° 


FREE 


n 


> 


R(1-2 R ~ 1 ) 
H(1-2 R_1 ) +R-1 
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Proof Note that no information sequence with a string of m 
or more all-zero blocks can produce the minimum free weight 
codeword. Hence if 


FM,d) < 1 


there exists at least one code with 


^FREE 


> d. Let F 


ma ? „ F(£, d) . Then if -L F <1, there exists at 

1 <JL<t max max 

~ - max 

least one code with cL, REE > 3. Alternatively, if d^^ is 

the smallest integer such that b F i> 1, then there 

3 max max 

exists at least one code with free distance greater than or 


equal to 


First an upper bound on F „ will be obtained. Since 

max 


£ /N(m+t)^ 2 N( m+ t) H (^5-) 


2 K - 1), 2 

2K+Nm 


,N(m+t) H 


X(N-K) 


d 

N (m+O 


Therefore an 


upper bound on F can be obtained by maximizing N (m+t) H 

max 


d 

N(m+i) 


) -i(N-K) 


Let L be the value of h which maximizes 


this expression. By setting the derivative of ^N(m+£) H 
( n (m+13 ) equal to zero and solving for -L, it can be 


shown that L 


N(1-2 R " 1 ) 


m. Therefore 


F < T(2 K -1) 2 
max - 2K+Nm 


I^fcr h(1 - 2R_1 ) 


N(1-2 R “ 1 ) 


m(N-K) 


2 
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Therefore if 


t T (2 K -l) 2 R-l 

max 2 1-2 


d H (1-2 R_1 ) 


2K + Nm 


< 1 , 


d(N-KV 
N (1”2 R_1 ) 


m(N-K) 


then '''max F max < Hence if 


that 


dpRE E is the least integer such 
^FREE 


t T (2 K -1> 2 2 - -R-l H( 1-2 R ' 1 > 


max 


1-2 


,2K + Nm 


d (N-K) 
FREE 


N(1-2 R " 1 ) 


> 1 , 


m(N-K) 


then there exists at least one code with free distance greater 
than or equal to dp^g. 

Therefore the least integer dp REE must be found such that 


(M+t )t ( 2 K -i) 2 2 1"2 R “ 1 

max max ' z 


dpRKB H(1-2 R ~ 1 ) 


d FREE ( N ” K ^ 
N(1-2 R_1 ) 


2 m(N-K) > 2 2K+Nm 


d 


_ log 
2 y 2 


(m + -t- ) 

max max 

d FREE^ K_R ^ 


v’ FREE p _ -I 

+ 2 log o ( 2-1) + H(l-2 ) 


2 " R-l + m(N-K) 

* * N(l-2 ) ^ 2 


1-2 

2K + Nm 


1o 9 2 ['“^max^max] + 2 + ® (N ' K > ' 


(1-R) dp REE 
( 1 - 2 R “ 1 > 


^FREE R , 

+ H (1-2 ) ^ 2K + Nm 

(1-2 R_1 ) 
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log. 


(m+'L ) -L 

max max 


m 


2 log 2 ( 2 K -l) 

+ — — r— — + (n-k) + 


m 


m(l-2 R " 1 } 


H(1-2 R " 1 ) -(1-R) 


£ — + N 
m 


(N-K) + 


FREE 


m 


^FREE 


m 


H (1-2 


R-l 


H(1-2 R “ 1 ) - (1-R) 


( 1-2 R ” 1 ) 


) + R-l 


(1-2 R_1 ) 


£ N 

^ K a s m -* 00 


as m -*■ 00 


S’REE > K(1-2 R ~ 1 ) 


m 


H(1-2 R_1 ) + R-l 


as m — °o 


^FREE R(1-2 R ” 1 ) 
lim 1 r > 


m — oo 


H (1-2 R-1 ) + R-l 


This bound is plotted in Figure 4.5 along with Neumann's 
bound and the usual Gilbert lower bound on d^. It is inter- 
esting to note that the bound given in theorem 4.4 is ex- 
actly the same as Neumann's bound for R < 0.37 with 2R re- 
placed by R. 

2. An Upper Bound on Error Probability for Maximum 
Likelihood Decoding over a BSC 

Viterbi [ 2 ] has given upper and lower bounds on the re- 
liability function, E(R), for the best periodic code used 
with maximum likelihood decoding over a BSC. Theorem 4.4 can 
be used to obtain a lower bound on E (R) , i.e., an upper bound 
on error probability, which is better than Viterbi ' s bound 
for low rates and which meets Viterbi ' s upper bound on E(R) 


at R = 0. 
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Fig. 4 . 5 . 
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Assume that the length of the information sequence is 

Z and let the period of the code be T. There are then 2 

possible transmitted sequences beginning at time u, 0 < u 

< T. Now label the non-zero codewords beginning at each 

time u from 1 to 2 -1 and let w^(u) be the Hamming weight 

til 

of the i non-zero codeword beginning at time u, 0 < u < T. 

Gallager [ 25 ] has shown that the probability of error 
for an R = k/n block code used with maximum likelihood de- 
coding over a BSC with digit error probability p is bounded 
by 2 k -l 

P < 
e 

i=l 

where p' =2 -yfp (l-p) and w^ is the Hamming weight of the 
i tk non-zero codeword. Hence for periodic convolutional 
codes this bound becomes 


l 


W M 


log p 1 
e 


( 112 ) 


< T 
e 



i=l 



log e p, J ; 


(113) 


where w! = min | w 1 .' (u) ! . Equation (113) can be rewritten as 

1 0<u<T ( 1 


l-l 

P e < T l 2 J K e W j log e P ' 


(114) 


j=0 

where w. is a lower bound on the Hamming weight of all the 
J 

non-zero codewords over their last m+j+1 blocks. Note that 
the first -t+m-m-j-1 = -t-j-1 blocks of each non-zero codeword 
agree with the first <t-j-l blocks of the transmitted code- 
word, and hence the distance contribution of these blocks is 
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omitted in (114) . 

Since the best code satisfies any lower bound on mini- 
mum distance, the weight of the last m+j+1 blocks of each 
non-zero codeword in the best code can be underbounded by 
both the usual Gilbert lower bound on dp D (which also applies 
to dp^g since d^ < d^^) and for large enough T by the 
lower bound on dp REE given in theorem 4.4. Hence 


w . = max 
J 


X'N(m+l), XN(j+m+l) 


^FREE 

where X' = N(m+1) = 


R (1-2 R ~ 1 ) 
H(1-2 R “ 1 ) +R-1 


and X. = 


FREE 
N (m+1) 


(115) 


= H _1 (1-R) . 


Let be the value of j at which X' N(m+1) = XN(j4m+l). 

Then for j < j^, X 1 N(m+1) is the dominant term in (115) and 
for j £ j^, XN(j+m+l) is the dominant term in (115). Let 


Q 


ii 

m 


Theorem 4.5 For maximum likelihood decoding over a BSC, 

-n. 


lim 
m — oo 


P < e 1 A 
e 


-QR-X 1 log p' 
e 


for the best periodic code with large enough T, if K log e 2 + 


X 1 N log p ' 
e 

Q+I 


< 0 . 


Proof 


„ . log T 

P < e 
e 


l-l 

ilj 2 JK e W j l0g e P ' 


X - 1 

I 

L j=0 


j=0 


jK e X'n A log e p' + l 


J=J 


0 


jK X N(j+m+l) log P 
e e 


2 


2 
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r ^'V 1 

P < e log e T 2 V e X ' n A log e p ' + I 2 <J ° +1>K 

e n i =0 

Xn ( j +i+m+l) log p' _ 

“ u 0 j ^2^ 

P e < e log e T [e J 0 NR ' e X ' n A log e p ' ° e <V 1,NR ’ 

XN(j +i+m+l) log p' i=0 

e J 0 e , 

where R 1 = R log 2 is in nats. Since )v'N(m+l) = XN(j +m+l) , 

0 U 


P e < e log e T [e 3 0 MR ' * +£ 


e<V i,NR ' 


e K ' n A (1 + > l°9 e P' 

p < e log e T + J o NR ’ + K ' n A l 0 g e P 


A * n ( . ,± . i ) log p' 
e A j +m+l» 3 e r 




« r r 0 

1 +1 


lo< 3 T j 0 


P < e" n A ^ ~ n 


i (NR ' + 7 - -log P') 

5 j 0 +m+l e 


R' - X Mog p 


Itj -1 

* : r\ 


J r -t.-j.-l 

lim P e < e -n A <- QR ' - V1 °9e p,) 1 + £ 

m ■ ■ OO L . ' 


e i(Klog e 2 + ^ log e p') 


If <Klog e 2 + 5 ^ log e p') < 0 , 

lim P < e-V- QR ' - X,log e p,) fl + V 
m-*oo * L A 


i<Kiog e 2 + l°g e P'>J 

.... V ilKlog 2 + A? 


But ) e 


log p 1 ) 


Q+l e - 


(Klog 2 + log p') 

e e U+J. e 


when (Klog 2 + g-£j- log p') < 0. Hence 
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OO \ i vr 

e i (K log e 2 + l°g e P' ) 


I 


=il + 


i=0 


1-e (K log e 2 + oil lo g e p,) 


by the geometric series argument. Therefore 


lira p < e “ n A ( " Q R ' “ X ' lo V> 


m-*^o° e 


2 + 


l-e (K log e 2 + ST log e p '> 


Let 


2 + 


l-e (K log e 2 + oil los e p,) 


= B, a positive constant 




depending only on R and p (Q == -y- 1 - 1 for large m), where 
3 < B < oo . Then 

log b 

_ n ( _ £_ - Q R'- X' log p') 

lim P < e n A { n A , 

m ► oo e 

lim P < e‘ n A( ' Q R ' ' V log e p,) ,| 

m-^oo e 


Corollary 4.3 For maximum likelihood decoding over a BSC, 


lim E (R) > -Q R’ - X' log p' 

m— ►oo e 


ir X'N log p' 

K log 2 + — — — < 0, where E (R) = - -i- log P is 

e Q+l n A e e 

the reliability function for the best periodic code with 

large enough T. I . 

y k' N log p« 

Let R q be the value of R such that K log e 2 + - — — — — = o. 


Figures 4.6, 4.7, and 4.8 compare the bound of corollary 4.3 
with Viterbi ' s upper and lower bound on E (R) for all R < R Q , 
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= bound of corollary 4,3. 


Fig. 4.7. A comparison of the bound of corollary 4.3 with 
Viterbi ' s upper bound (VU) and Viterbi’s lower 
bound (VL) for p = 0.10. 


□do 



.1 

0 

A 

Pig. 



bound of corollary 4.3. 


4.8. A comparison of the bound of corollary 4.3 with 
Viterbi’s upper bound (VU) and Viterbi* s lower 
bound (VL) for p = 0.40. 
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with p = 0.01, 0.10, and 0.40, respectively. In each of 
r E (R) 

these figures q is plotted against ~ — , where C = 1+p log 2 

p + (1-p) log (1-p) = 1 + H(p) is the capacity of a BSC. 

R 

Table 4.1 gives values of R Q , C, and ^ for p = 0.01, 0.10, 
and 0.40. 

It is easy to see from Figures 4.6 and 4.7 that the 
bound of corollary 4.3 is superior to Viterbi ' s lower bound 
on E (R) for low rates and for low values of p. For higher 
values of p, near p = 0.50, Viterbi' s lower bound on E(R) 
coincides with his upper bound on E(R) , and hence no improve- 
ment is possible, as can be seen from Figure 4.8. Note also 
that the bound of corollary 4.3 meets Viterbi 1 s upper bound 



oo 


( 121 ) 
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TABLE 4.1 



p = 0.01 

p = 0.10 

p = 0.40 

R o 

. 37 bits 

.19 bits 

.013 bits 

c 

.92 bits 

.53 bits 

.03 bits 

R o 

c 

.40 

.35 

.45 
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Applying L 'Hospital's rule. 


E (R) 
u 


R=0 


1+ ( 

2 V pU-piJ 17 * 


log e 

1/s 

(2 yj p(l-p) 

)*l/s 2 

(s 

. 1/s 2 

! yj p(l-p) ) 

1/S 


.(2 V P(l-P) ) 


1/s 


(122) 


00 


1+ ( 2 >/ p(l-p) ) 1/S 
= - 1/2 log ( 2 Vp (I-P) ) • 


i°g e (2 '\fp(i-p))|s =°o 

(123) 


(124) 


Therefore E (0) = E_ (0) . 

u ij 


3. Upper Bounds 

McEliece and Rumsey [l8] have shown that for R = ~ 


systematic binary fixed codes 

d log n„ 

FREE x 1— R y 2 E 1 

- < — + — : + 


n 


2 (m+1) 


(125) 


n, 


E 


if 


, , — ~ > N, where n = (N-l) (m+1) is the number of 

j.+xog 2 “ E E 


parity digits in one constraint length of transmitted digits. 


n 


It can easily be shown that 


E 


1+l0 9 2 n E 


> N for some finite value 


cif m. For instance, if R = 1/2, (125) holds for all m ^ 8. 
Hence 


lim 
m — 00 


FREE 1-R 
n, < 2 


(126) 


for all R = ~ systematic binary fixed codes. It can be shown 
that this result extends to systematic binary fixed codes of 
all rates. Equation (125) then becomes 
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dFREE . i-B „ ' Log 2 n E ( K+1 , 
n A ^ 2 + 2K (m+1) + 2n A 


log 0 n 

2 E 


^REE l^R « a (K+2) 

n A < 2 + 2K(m+l) + 2n A 


for (N-K) > 2 


for (N-K) = 1 


(127) 


(128) 


if 


”B 


K + log 2 n E 


> 


R 


where n^, = (N-K) (m+1) = n A (l-R) . 


Again equation (126) results in the limit as m— *~°o. 

In order to extend this result to non-systematic binary 
fixed codes, it is convenient to make the following defini- 
tions. 


Definiaion 4.1 M . = max j degree 

J l<i<K ( 


G i <J) w 


, l<j<N, 


for those fixed codes whose matrix of generator functions 
G(D) contains only polynomial elements. j 


Mj is called the constraint span of the j 


,th 


encoded sequence. 


Note that m = max 
l<j<N 

Definition 4.2 


M . 
J 


N 


+ N is the number of transmitted 


"E = I M j 

j=l 

digits that can be affected by a non-zero information block 

x_ for fixed codes with constraint spans M., j = 1,..., N. I 
~~V j 

This definition of n^, is slightly different from the defini- 
tion given by McEliece and Rumsey. 

Theorem 4.6 For any fixed convolutional code. 


n. 


^REE 


< 


2 + 2 r 1o 9 2 **£ + 2 if 


n. 


E 


K 


+ log2 n . 




E 


R 


Proof Consider all information sequences of length h. The 
average weight of a codeword is 
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Nh + Y m . 
L-* 3 


Since is less than or equal to the average weight of 


all non-zero codewords. 


Nh + 


L m j 


^FREE - 2 Kh -I 


Nh + 


^FREE 


y m . / Nn + y m . 

j=i 3 if j=i j 




Nh n E” N Kh Nh 

Choose h such that + ~ — <2 -l. Then d ___ < - 5 - 

n E _ N FREE 

+ ~2 + 2 * 


FREE 


Now a more explicit way of choosing h will be derived, 
-u JSC (h-1) . _ . oKh 


Suppose h is chosen such that 2 


. . ,Kh 

< n E s 1 Then 


n„-N 


Nh 2 -N 

< ~T + 2 


n E x 2 Kh n E 1 

If > p / then > > — since Kh-K < log 

K+log 2 n E R Kh K+log 2 n E R 2 

~Kh 

n E - This implies that h < . Therefore 

Nh n E-N 2 Kh_1 + 2 Kh " 1 - § = 2 Kh - “ < 2 Kh -l 
2 + 2 < 2 

since N i> 2 except for the trivial R = ~ = 1 codes. 

n E _ 

Consequently, for — — > if h is chosen such 

K+log n R 

2 E 

that 2 K ^ ^ < n„ < 2 K ^, it is also chosen such that 

x!i 

Nh n E _N Kh 

— + T~ < 2 - 1 - 


Finally, since h < 


K+log 2 n E 
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H "e' N ! °E lo S2 n E 

^REE < 2i (K + log 2 V + T~ + 2 = T + “IT - 

n _ 

Corollary 4.4 lim d_ . E 

m _^oo FREE < 2 . 



Proof The last two factors in theorem 4.6 become neglig- 

n 

ible compared with as m — *-00 . Also, it can be shown 


that 


n- 


E 


K + log2n E 


> 


R 


for all R as m — *-00 . 


Note that theorem 4.6 and corollary 4.4 are completely 
general, i.e., they apply to both systematic and non-systema- 
tic binary fixed codes of all rates. In the systematic case, 
these bounds are exactly the same as the bounds of (125) and 
(126) except for the slightly different definition of n E . 

Note that for systematic codes, = M 2 = ... = KL= 0. 

Hence corollary 4.4 indicates that more free distance may be 
available for non- systematic codes than for systematic codes 
of the same memory. The bound of corollary 4.4 for non-system- 
atic codes is shown together with the bound of equation (126) 
for systematic codes in Figure 4.9. 

For periodic codes, the constraint spans Mj must be re- 
defined as 


M . = max max 
0<u<T 0<i<m 


4-T«> 

i | the j column of Gh (u) ^ 0 

(129) 


Again m = max 
l<j<N 



Then by a slight modification of the 


proofs, theorem 4.6 and corollary 4.4 can be shown to hold 


for periodic codes. 
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Figure 4.10 sums up the results of Section IV. C. by 

plotting together the usual Gilbert lower bound on <3^, , and 

the bounds of equation (110), theorem 4.4, equation (126), 

and corollary 4. 4. Note that the lower bounds on d of 

FREE 

equation (110) and theorem 4.4 for non- systematic codes 

cross the upper bound on cl of equation (126) for sy sterna- 

F RES 

tic codes. This fact guarantees that more free distance is 
available with non- systematic codes than with systematic 
codes. 

Example 4.1 For R = 1/2, m = 3, the best fixed systematic 

(1) ( 2\ 3 

code, viz. the code with G (D) =1, G ' (D) = 1 + D + D , 

has dp REE = 4. For this code = 0, = 3, and n R = 5. 

Now consider the R = 1/2 non- systematic fixed code with m = 3 

(1) 3 (2) 9 3 

and G (D) = 1 + D + D , G (D) = 1 + D + D . This code 

has <3p REE = 6, = 3, = 3, and n E = 8. Hence for m = 3, 

non-systematic codes are clearly superior to systematic 

codes, j 

Example 4.2 In this example a fixed code which meets the 
bound of theorem 4.6 will be presented. For R = 1/2, m = 4, 
let G (1 * (D) = 1 + D + D 2 + D 4 , G^ 2 ' 1 (D) = 1 + D 2 + D 3 + D 4 . 


Then 


n E L M j + 2 - 10 and R + x n “ 1 + log 10 > R “ 2 ' 
j=l * * * 

n E 1 

so the bound is valid and yields < " + 2R log 2 n E 

+ 1/2 = 5 + log ^ 10 + 1/2 = 8.8, which implies that d pREE ^ 8. 
It can easily be shown that d pREE = 8 for the above code. | 



bound of corollary 4.4 
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The fact that non-systematic codes can produce more 
free distance than systematic codes is important when se- 
quential decoding is being used. Recall that cL is a 

FREE 

more appropriate distance measure for sequential decoding 
than dp D or d^. Hence a sequential decoder should exhibit 
a lower probability of decoding error for the best non-system- 
atic code of a given memory than for the best systematic 
code of the same memory. Happily, Bucher [26] has obtained 
theoretical results which indicate the possibility of 
achieving lower error probabilities with non-systematic 
codes than with systematic codes. Experimental verification 
of this fact is given in Chapter VI. 

E. A Gilbert Lower Bound for an Easily Instrumented Sub- 
class of R = 1/2 Non-systematic Codes 


The usual Gilbert lower bound argument states that there 


must exist at least one code with 


> d if d is the least 


integer such that 


no. of codewords with 


j =0 I x Q t Q. of weight j 


no. of codes which can 
x generate a particular £ 
codeword of weight j 


total no. 

of codes (130) 


Consider the class of R = 1/2 fixed non-systematic codes 
with G (2) (D) = D + G (1) (D) and g 0 (1) = g Q (2) = l. For any 

information sequence and any G^ (D) with g ^ = 1/ each 
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codeword y always has exactly one 1 in Hence the number 

of first constraint length code digits which can be chosen 
independently is n -1 = 2m+l. 

For any information sequence x whose transform is x(I$ , 
the sum of the transforms of the two transmitted sequences 


is given by 

x(D) G (1) (D) + x(D) 


D + G (1) (D) 


y (1) (D) + y (2) (D) 


(131) 


Hence 


and 


Dx(D) = y (1) (D) + y (2 * (D) 


x _ v U) + y (2) 

X -- - y^j.1 + Y j+1 


/ j 0 / 1 / • • • 


and x 


m 


j - j +i 

Therefore a particular choice of [y] fixes fx 

m 1 m— 1 

can be chosen arbitrarily. Also, for any particular choice 

of [x] and [y] , the matrix of generator functions G(D) : 


(132) 

(133) 


m 


[ G^ (D) D + G^ (D) ] is fixed. Hence only two codes can 


produce any specified choice of [y] 


Note also that there 


m 


are 2 m possible choices for G^ (D) and therefore 2 m possible 


codes. Equation (130) then reduces to 
d-1 


l 

j=0 


2m 

j 


x 2 > 2 


m 


(134) 


But 


or 


m 


d-1 

y 

/ 2m \ 

03 

VI 

L 

V J / 


j=0 

(2m) H 1 

( a-i\ 


(‘2m~ ) 

lim 

,-s-OO 

« 1 

k n A J 


( 2m) H 


> m~l 


m 


and (134) becomes 


(135) 


This proves the following result. 


(136) 
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Theorem 4.7 For R = 1/2 non- systematic fixed codes with 

(2) (1) 

G (D) = D + G (D) , there exists at least one code such 


that 


^FD 

lim £[■ — > . 110, 

m — *~co A 


Note that theorem 4*7 is exactly the same as the 
Gilbert lower bound on d^ for R = 1/2. Therefore it is guar 
anteed that this class contains good codes. Since non- system 
atic codes are better for sequential decoding than systematic 
codes, codes of this type should perform very well. Another 
property of these codes is that they can be simply instru- 
mented and that they possess the principal advantage of 
systematic codes, the so-called "quick look" capability. The 
instrumentation and "quick look" capability of these codes 
will be explained in detail along with their construction, 
simulation, and performance in Chapter VI. 



V. Some Results on Free Distance 


A. Bounding the Length of the Information Sequence Which 
Produces the Shortest Minimum Free Weight Codeword 
Property F4 of free distance implies that a finite num- 
ber of blocks of information digits are needed to produce the 
minimum free weight codeword for non-systemati:c fixed codes. 
And lemma 4.2 shows that a finite number of blocks are needed 
for non- systematic periodic codes. 

For the special case of R = 1/2 non-systematic fixed 
codes with (d) and (D) relatively prime polynomials 

an improved bound on the length of information sequence needed 
to produce the minimum free weight codeword can be obtained. 
Since G^ (d> and G^ (d) are relatively prime, there exist 
polynomials A(D) and B(D) of degree less than m such that A 
(D) G^ (D) + B(D) G^ (D) = 1 [27 ] . Hence for any informa- 
tion sequence x whose transform is x(D) , 

x(D) A(D) G^ (D) + x(D) B(D) G^ 2 ^ (D) = x(D) [a(D) G* 1 * (D) + 

B (D) G (2) (D)] (137) 

= x(D) . (138) 

Since an information sequence capable of producing the 
minimum free weight codeword cannot have any span of m con- 
secutive zeros, the minimum free weight codeword cannot have 
any span of 2m~l consecutive all-zero blocks. Therefore at 
least one 1 must be produced in every 2m-l encoded blocks. 
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But every R = 1/2 non- systematic fixed code has dj. REE at 
most 2 (m+1) , and 2 (m+1) I's must be produced in the minimum 
free weight codeword within (2m+l) (2m-l) +1 blocks. Hence 


d FREE = r 4m 2 -m-l * ^ 139 ^ 

Note that the bound of equation (139) is derived in a 
similar fashion to the bound of property F4 since the matrix 


G~ 1 (D) 


A(D) 

B(D) 


is a zero-delay feedforward inverse for 


G(D) whose polynomial elements have maximum degree M = m-1. 

A similar argument can be used to derive the following bound 
for fixed systematic codes of all rates: 


^FREE = r (N-K) (m+l)m ‘ (140 ' 

These bounds appear to be very weak since in practice 

the minimum free weight codeword is almost always produced 

by the first m+1 blocks of information digits. This remains an 

important problem since a tight bound would greatly simplify 

the calculation of <1 . The difficulty in proving a tight 

FREE 

bound stems from our lack of knowledge about the weights of 
products of polynomials. However, for some special cases, 
tighter bounds can be obtained. For instance, if d^ D is the 
reverse feedback decoding minimum distance and if 


d FD + ^FD > r i 
for any i, then 

^FREE = r 2m * 


(141) 
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This follows from the fact that any information sequence with 
some x^ / 0, i > 2m, produces a codeword with weight at least 
dp D over the first m+1 transmitted blocks and weight at least 
d^ D over the last m+1 transmitted blocks. Theorem 2.4 then 
yields (141) . If the code is reversible, then (141) holds if 
2dp D > r for any i. 

Also, for almost all fixed and periodic systematic codes, 
lemma 4.2 can be modified as follows j 

d FREE r (3I) m ' (142) 

where I is the least integer such that r^ < d^ + Id^ for 

any i, d^ satisfies equation (105) with equality in the 
fixed case and satisfies theorem 4.1 with equality in the 
periodic case, and d^ satisfies equation (103) with equal- 
ity. 


As noted earlier, Neumann [ 11 ] has suggested that the 
correct bound on the length of information sequence needed 
to produce the minimum free weight codeword is m+1 blocks, 
i.e., dp REE = r * Unfortunately, the following counter- 
example disproves this conjecture. 

MV 9 4 5 £> in in 

Example 5.1 Let G y ' (D) = 1 + D + D + D + D + D + D X 

/ 2) 2 3 4 8 9 

and G v ' (D) = 1 + D +D +D +D +D for an R = 1/2 non- 

, . 2 5 

systematic fixed code with m = 11. For x(D) = 1 + D + D + 

8 9 10 11 12 (1) , v . . (1), 23 

D+D+D + D + D , y (D) = x(D) G (D) = 1 + D 

(2) (2) , 3 6 7 8 9 10 

and y (D) = x(D) G (D) = 1 + D + D + D + D + D + D + 

21 , v 
D and the weight of this codeword is 10. But for any x(D) 

such that Xq / 0 and degree £x(D)J < 11 = m, the weight of 
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the codeword produced by x(D) is at least 11. Hence dp REE 

/ r .| 
m 

Note that G^ (D) and (D) in example 5.1 are relatively 

prime, so the conjecture does not hold even when the gener- 
ator functions are relatively prime polynomials. 

The same conjecture has also been made for systematic 

codes only. However consider the following counterexample. 

Example 5.2 Let G (1) (D) = 1, G* 2) (D) = G (3) (D) = 1 + D + 

2 4 6 7 8 

v D + D + D + D + D for an R = 1/3 fixed systematic code 

with m=8. For x(D) = 1 + D + D 3 + D 6 + D 8 + d 9 , y (1) (D) = 

x(D) G^ (D) = 1 + D + D 3 + D 6 + D 8 + > Y^ (D) = Y^ (D) = 

x(D) G^ (D) = x(D) G^ 3 ^ (D) = 1 + and the weight of this 

codeword is 10. But for any x(D) such that x Q / 0 and degree 
|x(D)J < 8 = m, the weight of the codeword produced by x(D) 
is at least 11. Hence dp REE / r m . | 

It is interesting to note, however, that no counter- 
examples to this conjecture have yet been found for R = 1/2 
systematic codes. The author has been able to find some 
codes for which an information sequence with degree greater 
than m produces a codeword with weight equal to r , but none 
with weight less than r^. The difficulty may lie in the fact 
that very long codes are needed to provide counterexamples, 
and distances are very difficult to calculate for long codes. 
Also, no counterexamples have been found for systematic codes 
of rate other than 1/2 whose generator functions are relative- 
ly prime polynomials. 
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The difficulty in finding counterexamples to this con- 
jecture leads one to believe that the true bound is very 
close to r , perhaps at least for codes whose generator 

functions are relatively prime polynomials. Hence it is 
also likely that r m is always very close to the actual free 
di stance. 

B. Calculating d^ REE 

Unfortunately, there can be no simple, general method 
of calculating dp REE until a tight bound is obtained on the 
length of information sequence needed to produce the minimum 
free weight codeword. Then <3p REE can be calculated simply 
by computing the minimum row distance over the bounded 
length of information sequence. However there are many 
tricks which can be used to find or to closely approximate 

^FREE* 

Since d^ ^ ^free - r i ^ or ^i an< ^ r i can ■* 3e 

successively computed. If at some point j, dj = r . , then 

^FREE = d j = r j‘ Property F4 showed that = d. = r._ m 

for some finite j if the encoder has a feedforward inverse. 

Sometimes the free distance of the reverse code, i.e., 
the code whose generator matrix is described in definition 
2.15, is known. If so, the free distance of the original 
code is the same as that of the reverse code. This follows 
from the fact that each codeword in the reverse code is the 
reciprocal of the codeword in the original code produced by 
the reciprocal information sequence, where the reciprocal 
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of the sequence z = Jz Q/ z ± , z 2 ,...J is taken to be . .., 
z^, z^ Zq J . Since the weight of a sequence and its re- 
ciprocal are the same, the set of codeword weights of a 
code and its reverse code are the same. Note that this is 
true only for d^^, and not for dp D or d^. 

Very good approximations to <3^ E can be found by 
computing r^ or d^ for as large an i as feasible. For in- 
stance, a computer program has been written for use on the 
Uni vac 1107 computer at the University Computer Center which 
calculates d,.., for R = 1/2 fixed codes in just a few minutes. 
This usually provides a very good approximation to 3p REE f op- 
codes with encoding memory less than about 50. d & ^ is a 

lower bound on <3^ an< ^ a reasonable upper bound is usual- 
ly known from the weight of the generator or some short low 
weight codeword. Hence d^ is often known to be exactly 
^FREE' an< ^ ^ n0t ■*•*" eas ¥ to ma ^ e a close approximation. 
The values of and bounds on < 3 RREE given in Appendix A were 
arrived at in this manner. However, for other rates and 
longer codes, good approximations to dp REE become harder to 
make. 

Example 5.3 Consider the m = 71, R = 1/2 fixed systematic 

(2) „ (2) „ (2) 


code whose generator sequence 


[ g 0,l ’ # g l,l ' ** * ' g 71 , 1 
= |9 q / g^,..., g^J is represented three digits at a time 
in octal notation, starting with g Q , as ^651, 102, 104, 

121, 022, 041, 101, 101 J . This code is known to have d^ = 
21. But since the weight of the generator is only 21, dp REE 
< 21. Therefore d pREE = d p>D = 21. This code is one which 
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will be constructed in Chapter VI. 


Consider the following m = 35, R = 1/2 fixed 


= [715, 473, 701, 


This code has d,. =18. But the codeword whose 

61 


systematic code with [g Q , g^, g 2 , . . . , g 35 J = [715, 473, 701, 
317 J . This code has d &1 = 18. But the codeword whose 
transform is (1+D) G(D) has weight 18. Therefore dp REE = 18. 
This code was constructed by Forney [28] and' is presently 
being used by NASA in its Pioneer satellite series. | 


The m = 35, R = 1/2 fixed systematic code with 
[g Q , g^,..., g^J = [653, 134, 307, 713j has d gl = 19. Since 
the generator has weight 22, 19 < < 22. This code is 

due to Lin and Lyne [ 29 ] . | 



VI. Constructing Good Convolutional Codes 
A. The Minimum Weight Construction Algorithms 

In this chapter various algorithms for constructing 

R = fixed binary convolutional codes will be given. In 
constructing good codes it is desirable that the complexity 
of the encoder be kept as small as possible. For example, 
if a code is being used to communicate information from a 
space vehicle to earth, the encoder is a hardware device on 
the space vehicle itself. The usual encoding circuit for an 
R = — canonic systematic fixed binary convolutional code is 
shown in Figure 6.1. Note that the number of two-input 
modulo-two adders required to implement this encoder is ex- 
actly w H (^) - N, where a is the generator. Therefore minimiz- 
ing w H (c[) for a given distance and constraint length mini- 
mizes the number of modulo-two adders in the encoder realiza- 
tion. All the codes presented in Section VI. A will exhibit 
this property, i.e., for a given distance and constraint 
length, w„(g) will have its minimum possible value. As in 
Chapter II, j^gj . will be used to denote the first (j + 1)N 
entries in g. 

1. An Algorithm for Finding Good R = 1/2 Convolutional 
Codes 

In this section, a simple algorithm will be given which 
will be shown to produce good R = 1/2 canonic systematic 
fixed binary convolutional codes for all m <! 71. First a 
statement of the algorithm is given and then several inter- 
esting properties of the codes produced are shown. (For 
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( 2 ) 

convenience, let g^ = <3y) 

Algorithm Al 

(0) Set g Q = 1, d Q = 2, and j = 1. 

(1) Set g^ = 1. 

(2) Compute d^. If dj > dj_^, go to (4) . 

(3) Set g^ =0. 

(4) If j = m, stop. Otherwise, set j = j+1 


/ • • • / 


m. 


and go to (1) . 

Property Al-1 w H ( [a] _.) = d for j =0,1 

Proof w ( [a] ) > d . by property C3 of the column dis- 

H J j J 

tance. However, since w^Gq) = d^ = 2 and since g^ is per- 
manently set to 1, i.e. , w H ( [a] .) is increased by one, if 

and only if d. > d. 1# w^( [a] .) d.. Therefore w R ( [a] ) 

V 1 


Since properties Cl and C3 of the column distance require 

that wij ( a ) = w„( [al ) > d = d , property Al-1 en- 
H H I J m m FD 

sures that w H (, g) is minimal and hence the resultant code 
requires the minimum number of modulo-two adders in its en- 
coding circuit. 


Property Al-2 If g = 1, then g j+1 = 0, for every j ^ 0. 
Proof Assume g^. =1, j £ 1 (note that algorithm Al sets 

g = q =1). Then set g. =1. The information sequence 
^0 1 J+1 


[x] = [x (1) x (1) ...x. , (1) x. (1) x U) 1 = fi o 

j+i L° 1 j -1 j j+i J L' — ' 

0,1,0 J always produces a codeword with dj + ^ = d^. There- 
fore algorithm Al will set = 0. j 
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Property Al-2 allows us automatically to add a 0 to Jc[] 
after adding each 1 beyond g Q . This permits a shortcut to 
reduce the number of times steps 1 and 2 must be applied to 
reach a given length code. 

Property Al™3 Let <2 be the generator obtained by using 
algorithm Al. Let 3 ' / g; be the generator of any other 
R = 1/2 canonic systematic fixed code of the same length 


i.e. , such that 


such that w R ( [a'j ) = d V. / j = 0 , 1 , ... , m, i.e., such that 
each 1 in the generator increases the column distance by one. 
Then there exists a i_, 0 < i < m, such that d. > d' and 

0 0 Jo Jo 

d. ~ ^ . > i - 0 , 1 , 2 ,..., j ~ 1 
11 0 

Proof Assume the first point at which the two generators 


disagree, 0 < < m, has g^ 


< j < m, has g, = 0, g'. =1. Then dl = 

0 J 0 J 0 J 0 

But this is impossible, since if the column 


d. + 1 > d. . But this is impossible, since if the columi 
J 0 J 0 

distance can increase at Jq algorithm Al would make g^ = 1. 
Therefore, the first point at which the two generators dis- 
agree must have gl = 0 , g. = 1 , and hence, d. > dl . | 


Property Al-3 shows that any other algorithm for generating 

R = 1/2 fixed canonic systematic convolutional codes which 

increases the column distance by one each time a 1 is added 

to the generator differs from algorithm Al in that such 1 1 s 

are not always added at the first opportunity. Note also 

that in the computation of step ( 2 ) , if d . > 3 . n > then 

J J-l 

dj = d. +1, and if g. is set to 0 in step (3), then d. = d. 

J j-l j j-1 

and that the codes obtained from algorithm Al exhibit the 
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"nested" property, i.e.. 


^ m^+1 




Algorithm Al was programmed on the Uni vac 1107 com- 
puter at the University Computer Center. The most diffi- 
cult part of algorithm Al to program is the computation of 
dj in step (2). This was done by using a sequential-de- 
coding-like algorithm suggested by Forney [28] . The flow 
chart for this algorithm, called SEAL, is shown in Figure 
6.2. The flow chart for algorithm Al is then shown in 
Figure 6.3. 

The codes obtained from algorithm Al are compared with 
Bussgang's [ 30 ] optimal codes and Lin and Lyne's [ 29 ] near- 
optimal codes in Table 6.1. Bussgang's computer search for 
optimal codes reached m = 15 before the amount of computa- 
tion became too large. Lin and Lyne carried their near- 
optimal search out to m = 20 (Forney [28] has extended this 
to m = 48) . Algorithm Al is sufficiently simple to allow 
hand computation out to m = 22 and it was extended to m = 

71 by computer. Table 6.1 also compares the codes obtained 
with the non-asymptotic Gilbert lower bound [ 30 ] , and it 
can be seen that the codes remain good out to m = 71. The 
adjoints of the codes obtained from algorithm Al, which 
are known to have exactly the same set of codeword weights 
over the first constraint length [ 30 ] , are also given in 


Table 6.1. 






















start 


WR = write 
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j=o 

di=2;gr1 


T = threshold 

W ® Hamming weight of path 



Fig. 6.3. Flow chart for algorithm Al. 
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TABLE 6.1 

COMPARISON OF R = 1/2 CODES 


j 

s j 

A 

g j 

d. 

3 

d G 


d 

B 

j 

g j 

g j 

d. 

J 

d G 

d 

LL 

0 

1 

1 

2 

2 

2 

2 

36 

0 

0 

13 

11 

14 

1 

1 

1 

3 

3 

3 

3 

37 

0 

1 

13 

12 

14 

2 

0 

1 

3 

3 

3 

3 

38 

0 

0 

13 

12 

14 

3 

1 

0 

4 

4 

4 

4 

39 

0 

1 

13 

12 

15 

4 

0 

1 

4 

4 

4 

4 

40 

1 

0 

14 

12 

15 

5 

1 

1 

5 

4 

5 

5 

41 

0 

0 

14 

13 

15 

6 

0 

0 

5 

4 

5 

5 

42 

0 

0 

14 

13 

15 

7 

0 

0 

5 

5 

5 

6 

43 

1 

0 

15 

13 

16 

8 

1 

0 

6 

5 

6 

6 

44 

0 

0 

15 

13 

16 

9 

0 

0 

6 

5 

6 

6 

45 

0 

1 

15 

13 

16 

10 

0 

0 

6 

5 

7 

7 

46 

0 

0 

15 

14 

16 

11 

1 

1 

7 

6 

7 

7 

47 

0 

1 

15 

14 

17 

12 

0 

1 

7 

6 

7 

8 

48 

1 

1 

16 

14 

17 

13 

0 

1 

7 

6 

8 

8 

49 

0 

1 

16 

14 


14 

0 

0 

7 

6 

8 

8 

50 

0 

0 

16 

15 


15 

0 

0 

7 

7 

9 

9 

51 

0 

1 

16 

15 


16 

1 

0 

8 

7 

9 


52 

0 

0 

16 

15 


17 

0 

0 

8 

7 

9 


53 

1 

0 

17 

15 


18 

0 

0 

8 

7 

9 


54 

0 

0 

17 

15 


19 

0 

1 

8 

8 

10 


55 

0 

1 

17 

16 


20 

1 

0 

9 

8 

10 


56 

1 

0 

18 

16 


21 

0 

1 

9 

8 

10 


57 

0 

1 

18 

16 


22 

0 

0 

9 

8 

10 


58 

0 

1 

18 

16 


23 

0 

1 

9 

9 

10 


59 

0 

0 

18 

16 


24 

1 

0 

10 

9 

11 


60 

0 

0 

18 

17 


25 

0 

0 

10 

9 

11 


61 

0 

1 

18 

17 


26 

0 

1 

10 

9 

11 


62 

1 

0 

19 

17 


27 

1 

0 

11 

9 

11 


63 

0 

0 

19 

17 


28 

0 

0 

11 

10 

12 


64 

0 

1 

19 

18 


29 

0 

1 

11 

10 

12 


65 

1 

0 

20 

18 


30 

0 

0 

11 

10 

12 


66 

0 

0 

20 

18 


31 

1 

1 

12 

10 

12 


67 

0 

1 

20 

18 


32 

0 

0 

12 

11 

13 


68 

0 

1 

20 

18 


33 

0 

0 

12 

11 

13 


69 

0 

1 

20 

19 


34 

0 

0 

12 

11 

13 


70 

0 

0 

20 

19 


35 

1 

0 

13 

11 

14 


71 

1 

1 

21 

19 



d G ^GILBERT BOUND d LL ^LIN AND LYNE d B d BUSSGANG 
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An interesting, but as yet unsolved, question is 
whether algorithm A1 will continue to produce good codes, 
i.e., codes whose column distance increases linearly with j, 
as j becomes arbitrarily large. The amount of computation 
required by algorithm Al, because of the calculation of d^ 
in step (2) , appears to increase exponentially with increas- 
ing j , as it does in all known search techniques for finding 
codes. However, because of its simplicity, algorithm Al 
requires less computation than other known search techniques. 

2. Algorithms for Generating Good R = ^ and R = ^ Codes 

For rates R = ■■, N > 2, an algorithm is sought for gen- 
erating codes such that d^ = w^( [s] j) , j =0,1,2,..., m, 
and 1 1 s are added to the generator at the first opportunity 


consistent with this constraint. Since there are now N - 1 

( 2) ( 3) (N) 

digits, viz. g^ , g^ ,..., g^ , to be specified in 
each block, there will not be a unique algorithm with the 


above property for N > 2. For example, for N = 3 the three 


following algorithms each result in a code such that d^ = 
w H ( [a] j)and "ones" are added to the generator at the 
earliest opportunity. For N = 3, it is well known [ 29 ] that 


dj < dj ^ + 1 so that it is unnecessary to test the choice 

g..^ = g.-,^ =1 since the column distance can never in- 
jl jl 

(2) (2) (3) 

crease by 2. (For convenience let g.. = g . and g., 

= gj <3 >.) 
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Algorithm A2 


(0) 

q . „ (2) 

Set g Q 

- 9 0 <3 ’ = V 

= 3, 

and j = 1 

(1) 

~ „ (2) 
S 0t cf j 

-i. 9j <3) =o. 




(2) 

Compute d. 

. If d. > d. 

go 

to 

(6) . 

(3) 

Set g . ^ 
J 

= 0, ^ Jl. 




(4) 

Compute d. 

• If d j > Vr 

go 

to 

(6) . 

(5) 

„ (2 > J 
Set g . 

J 

(3 > n 
= gj = o. 




(6) 

If j = m. 

stop. Otherwise 

, set 

j 

= j + 1 


go to (1) . 






Algorithm A3 

Steps (0) through (5) are the same as in algorithm 

A2. 

(6) If j = m, stop. Otherwise, interchange steps 
(1) and (3) , set j = j + 1, and go to (1) . 

Algorithm A4 

Steps (0) through (5) are the same as in algorithm 

A2. 

(6) If j = m, stop. Otherwise, if d^ increased 
during step (2) , interchange steps (1) and 
(3) , set j = j +1, and go to (1) . If d^ 
increased during step (4) or remained the 
same, set j = j + 1 and go to (1) . 

The codes obtained from algorithms A2, A3, and A4 are 
shown in Table 6.2 and are compared to Bussgang's codes, 

Lin and Lyne's codes, and to the non-asymptotic Gilbert lower 
bound. Each algorithm was carried out to m = 35 by computer. 
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TABLE 6.2 

COMPARISON OF R = 1/3 CODES 






Algorithm A2 

Algorithm A3 

Algorithm A4 

j 

<0 

«LL 



■j" 

d. 

J 


(3) 

g j 

d. 

J 

■r 

(3) 

g j 

d. 

J 

0 

3 

3 

3 

1 

1 

3 

i 

1 

3 

i 

1 

3 

1 

4 

4 

4 

1 

0 

4 

i 

0 

4 

i 

0 

4 

2 

5 

5 

5 

1 

0 

5 

0 

1 

5 

0 

1 

5 

3 

6 

6 

6 

0 

1 

6 

1 

0 

6 

1 

0 

6 

4 

6 

7 

7 

1 

0 

7 

1 

0 

7 

1 

0 

7 

5 

7 

8 

8 

0 

1 

8 

1 

0 

8 

1 

0 

8 

6 

8 

9 

9 

0 

1 

9 

0 

0 

8 

0 

0 

8 

7 

8 

9 


0 

1 

10 

1 

0 

9 

0 

1 

9 

8 

9 

10 


0 

0 

10 

0 

1 

10 

1 

0 

10 

9 

9 

11 


1 

0 

11 

0 

0 

10 

0 

0 

10 

10 

10 

12 


1 

0 

12 

1 

0 

11 

0 

1 

11 

11 

10 

12 


0 

0 

12 

0 

1 

12 

0 

1 

12 

12 

11 

13 


1 

0 

13 

0 

0 

12 

1 

0 

13 

13 

11 

14 


0 

0 

13 

0 

0 

12 

0 

0 

13 

14 

12 

15 


1 

0 

14 

0 

1 

13 

1 

0 

14 

15 

12 

15 


1 

0 

15 

1 

0 

14 

0 

0 

14 

16 

13 

16 


0 

0 

15 

1 

0 

15 

0 

1 

15 

17 

14 

16 


1 

0 

16 

0 

0 

15 

0 

0 

15 

18 




0 

1 

17 

0 

1 

16 

1 

0 

16 

19 




0 

0 

17 

1 

0 

17 

1 

0 

17 

20 




1 

0 

18 

0 

0 

17 

1 

0 

18 

21 




0 

0 

18 

0 

1 

18 

1 

0 

19 

22 




1 

0 

19 

0 

0 

18 

0 

0 

19 

23 




0 

1 

20 

1 

0 

19 

0 

0 

19 

24 




0 

0 

20 

0 

0 

19 

0 

1 

20 

25 




0 

0 

20 

1 

0 

20 

1 

0 

21 

26 




1 

0 

21 

0 

1 

21 

0 

0 

21 

27 




1 

0 

22 

0 

0 

21 

0 

0 

21 

28 




0 

1 

23 

0 

1 

22 

0 

1 

22 

29 




0 

0 

23 

0 

1 

23 

0 

0 

22 

30 




0 

0 

23 

0 

0 

23 

1 

0 

23 

31 




0 

1 

24 

0 

0 

23 

1 

0 

24 

32 




0 

0 

24 

1 

0 

24 

0 

0 

24 

33 




1 

0 

25 

1 

0 

25 

1 

0 

25 

34 




0 

0 

25 

0 

0 

25 

0 

1 

26 

35 




1 

0 

26 

1 

0 

26 

0 

0 

26 


d G ^GILBERT BOUND ^LL d LIN AND LYNE d BUSSGANG 
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Again the codes are quite good and are considerably longer 
than other known good R = 1/3 codes. Note that the codes ob- 
tained from Algorithms A2, A3, and A4 exhibit about the same 
distance properties. Indeed it seems the many variations of 
the algorithm available for R = 1/3 will have little effect 
on the distance properties of the resulting codes. The two 
adjoints of each of the codes obtained from Algorithms A2, 

A3, and A4 are given in Table 6.3. 

Note that at m = 7, the code obtained from Algorithm A2 
has greater feedback decoding minimum distance than Lin and 

I 

Lyne's near-optimal code. It can be shown that this code 
meets the Plotkin upper bound [8] on feedback decoding mini- 
mum distance at m = 7. 

To generate R = 1/4 codes, g^* 2 *, g.^ 3 *, and g^ 4 * 
must be specified for each j, and it must be recognized that 
an increase of either one or two in the column distance for 


each j is possible. O’nly one algorithm will be given for 

generating R = 1/4 codes with the property that d. = w 

J H 

([g] ) and l's are added to the generator at the earliest 

J / <0% / AV / 

opportunity. (For convenience, let g ^ = g j ' g jl 

g / 3 >. ana gjl <« = gj < 4 >., 



= 0, i = 1, and 
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(2) Set =0, g.^ =1, i = 2, and go to (8). 

3 ( 2 ) ( 3 } 

(3) Set g. = 0, g. = 1, i = 3, and go to (8) . 

J (3) J 

(4) Set g^ =0,1 = 4, and go to (8) . 

(5) Set g (3) = l, g =0, 1=5, and go to (8). 

( 21 ( 31 

(6) Set g. = 1, g. = 0, i = 6, and go to (8) . 

3 (21 3 

(7) Set g^ =0 and go to (9) . 

(8) Compute d ^ . If dj = dj_^, 9° to ^ + 1) . 

(9) If j = m,stop. Otherwise, set j = j + 1 and 
go to (1) . 

Table 6.4 compares the R = 1/4 codes generated by Al- 
gorithm A5, Lin and Lyne's codes, and the non-asymptotic 
Gilbert lower bound. Algorithm A5 was carried out to m = 35 
by computer and again good codes were found. The three ad- 
joints of the code produced by Algorithm A5 are given in 
Table 6.5. Clearly properties Al-1 and Al-3, as well as the 
"nested" property, also hold for the codes of Algorithms A2 
through A5. 

3. An Algorithm for Generating R = 1/2 Codes with 
Large Free Distance 

Clearly, it is of considerable interest to find codes 
with known <3p REE , especially codes for which d^^ > d^- A 
slight modification of the preceding algorithms can be used 
for this purpose. Algorithm A6 indicates the necessary modi- 
fication of Algorithm Al. 
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TABLE 6.3 

ADJOINTS OF R = 1/3 CODES 




Alaorithm 

A2 


Alaorithm A3 

Alaorithm A4 


1st 

adj . 

2nd ad.j . 

1st 

adi . 

2nd adi . 

1st 

adi . 

2nd 

adi . 

j 



■J" 


g< 2> 

(3) 

g j 



/ 2 > 

•5* 

g < 2 > 

y j 

g< 3 > 

J 

0 

1 

1 

1 

i 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

0 

1 

1 

1 

0 

1 

1 

1 

0 

2 

0 

0 

1 

0 

1 

0 

1 

1 

1 

0 

1 

1 

3 

1 

0 

1 

1 

0 

1 

0 

0 

0 

1 

0 

0 

4 

0 

1 

0 

0 

0 

1 

0 

1 

0 

1 

0 

1 

5 

0 

1 

0 

1 

1 

1 

1 

0 

1 

1 

1 

0 

6 

0 

1 

1 

0 

1 

1 

0 

1 

1 

1 

0 

1 

7 

1 

1 

1 

1 

1 

0 

0 

0 

0 

0 

0 

1 

8 

1 

1 

1 

0 

1 

1 

1 

0 

0 

0 

0 

1 

9 

1 

0 

0 

1 

0 

0 

1 

0 

1 

0 

1 

0 

10 

0 

0 

1 

1 

0 

0 

1 

1 

0 

1 

1 

0 

11 

1 

0 

0 

1 

0 

1 

0 

1 

1 

0 

1 

0 

12 

0 

0 

1 

1 

0 

1 

0 

0 

1 

0 

1 

1 

13 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

14 

1 

1 

1 

1 

1 

1 

1 

0 

1 

0 

0 

1 

15 

1 

1 

0 

1 

0 

0 

0 

1 

0 

1 

1 

0 

16 

0 

0 

1 

1 

1 

1 

1 

1 

1 

0 

1 

1 

17 

1 

1 

0 

0 

1 

0 

1 

0 

1 

1 

1 

0 

18 

0 

1 

0 

0 

0 

1 

1 

0 

0 

1 

0 

0 

19 

0 

0 

0 

1 

1 

1 

1 

1 

0 

1 

0 

0 

20 

0 

1 

1 

0 

1 

1 

1 

0 

1 

0 

1 

1 

21 

1 

0 

1 

0 

1 

0 

1 

1 

1 

0 

0 

1 

22 

0 

0 

0 

1 

1 

1 

0 

0 

1 

1 

0 

1 

23 

1 

1 

1 

1 

1 

0 

1 

1 

0 

1 

0 

0 

24 

1 

0 

1 

1 

1 

0 

0 

1 

0 

0 

0 

0 

25 

1 

0 

1 

1 

1 

1 

0 

1 

0 

1 

0 

1 

26 

1 

1 

0 

1 

0 

0 

0 

1 

1 

1 

0 

0 

27 

0 

1 

0 

1 

1 

0 

0 

0 

1 

1 

0 

1 

28 

0 

0 

1 

0 

0 

0 

0 

0 

1 

0 

1 

0 

29 

1 

0 

0 

1 

0 

1 

1 

0 

0 

0 

0 

0 

30 

0 

0 

0 

0 

1 

0 

1 

0 

1 

0 

0 

0 

31 

0 

0 

0 

1 

1 

0 

1 

1 

0 

0 

1 

1 

32 

1 

1 

1 

0 

0 

1 

1 

1 

0 

0 

1 

0 

33 

0 

0 

0 

1 

1 

1 

0 

1 

1 

0 

1 

0 

34 

0 

1 

0 

0 

0 

0 

0 

1 

1 

0 

1 

1 

35 

0 

0 

1 

1 

0 

0 

1 

0 

1 

1 

0 

0 
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TABLE 6.4 

COMPARISON OF R = 1/4 CODES 


j 


(3> 

9 J 

g‘ 4 > 

y J 

a j 

d G 

^L 

0 

1 

1 

1 

4 

4 

4 

1 

1 

1 

0 

6 

6 

6 

2 

1 

0 

1 

8 

7 

8 

3 

0 

0 

1 

9 

8 

9 

4 

0 

1 

0 

10 

9 

10 

5 

0 

0 

1 

11 

10 

11 

6 

0 

0 

1 

12 

ll 

13 

7 

1 

0 

1 

14 

12 

14 

8 

0 

0 

1 

15 

13 

15 

9 

0 

1 

0 

16 

13 

16 

10 

0 

1 

0 

17 

14 

17 

11 

0 

0 

0 

17 

15 

18 

12 

1 

1 

0 

19 

16 

19 

13 

0 

0 

1 

20 


21 

14 

0 

1 

0 

21 


22 

15 

0 

0 

1 

22 


23 

16 

0 

0 

1 

23 



17 

0 

0 

1 

24 



18 

0 

1 

0 

25 



19 

0 

1 

0 

26 



20 

1 

0 

0 

27 



21 

0 

0 

1 

28 



22 

0 

0 

1 

29 



23' 

0 

0 

1 

30 



24 

0 

1 

0 

31 



25 

0 

0 

1 

32 



26 

0 

1 

0 

33 



27 

0 

1 

0 

34 



28 

0 

0 

0 

34 



29 

0 

0 

1 

35 



30 

0 

1 

0 

36 



31 

0 

1 

0 

37 



32 

0 

0 

0 

37 



33 

1 

1 

0 

39 



34 

0 

0 

0 

39 



35 

0 

0 

1 

40 



d G = 

^GILBERT 

BOUND 

= 

^LIN AND 

LYNE 
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TABLE 6.5 

ADJOINT S OF R = 1/4 CODES 



1st 

adjoint 

2nd adjoint 

3rd adjoint 


j 

(2) 

g j 


g ' 4> 

(2) 

S j 

g f’ 

(4) 

g j 

(2) 

g j 

g f ! 

■r 

0 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

0 

1 

0 

1 

1 

1 

1 

0 

2 

0 

1 

1 

1 

1 

0 

0 

1 

1 

3 

1 

1 

1 

1 

1 

1 

0 

0 

1 

4 

1 

1 

0 

0 

0 

0 

1 

1 

1 

5 

0 

0 

0 

0 

1 

0 

1 

0 

1 

6 

1 

1 

1 

1 

0 

1 

1 

1 

1 

7 

0 

0 

1 

1 

1 

1 

1 

0 

0 

8 

0 

1 

0 

1 

1 

0 

0 

0 

1 

9 

0 

1 

0 

0 

1 

1 

0 

1 

0 

10 

1 

0 

1 

0 

1 

0 

1 

1 

1 

11 

0 

0 

1 

0 

0 

0 

1 

1 

1 

12 

0 

0 

1 

1 

0 

0 

0 

1 

0 

13 

0 

1 

1 

0 

1 

0 

0 

0 

0 

14 

0 

1 

0 

0 

1 

1 

0 

0 

1 

15 

1 

0 

1 

0 

0 

1 

1 

1 

0 

16 

0 

1 

0 

0 

0 

0 

0 

1 

0 

17 

0 

1 

1 

1 

1 

1 

0 

1 

0 

18 

1 

0 

1 

0 

1 

1 

1 

0 

1 

19 

1 

0 

0 

0 

1 

0 

1 

0 

0 

20 

1 

1 

1 

0 

1 

0 

0 

0 

1 

21 

1 

1 

1 

1 

0 

1 

1 

1 

1 

22 

0 

1 

0 

0 

1 

0 

1 

1 

1 

23 

0 

0 

0 

0 

0 

1 

0 

1 

0 

24 

1 

0 

0 

1 

1 

1 

1 

1 

1 

25 

0 

1 

1 

0 

0 

1 

0 

1 

1 

26 

1 

0 

0 

0 

1 

1 

1 

1 

1 

27 

1 

1 

0 

0 

1 

1 

1 

1 

0 

28 

1 

0 

1 

0 

1 

0 

0 

1 

1 

29 

0 

0 

1 

1 

0 

1 

1 

1 

1 

30 

1 

1 

0 

0 

1 

1 

1 

1 

1 

31 

1 

0 

0 

1 

0 

1 

1 

1 

1 

32 

1 

1 

1 

0 

0 

0 

0 

0 

0 

33 

1 

1 

0 

1 

0 

1 

1 

1 

1 

34 

0 

1 

0 

1 

1 

0 

1 

0 

0 

35 

1 

1 

0 

0 

1 

0 

0 

0 

1 
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Algorithm A6 


A6 

(Assume L > m.) 




(0) 

Set g Q = 1, D q 

= 2, and j = 

1. 


(1) 

Set g. =1. 
J 




(2) 

Compute d . if 
L 

> D J-1' 

set D : 
J 

■ d L 


and go to (4) . 




(3) 

Set g . = 0 and 

3 

D j ■ “j-l' 



(4) 

If j = m, stop. 

Otherwise, 

set j = 

= j + 


and go to (1) . 





The following properties of the codes resulting from 
Algorithm A6 will be presented without proof, since the 
proofs are similar to those used to prove the properties of 
Algorithm Al. 

Property A6-1 w R ( [a ] ) = D. for all j. 

j 

Property A6-2 In the computation of step (2) , if > D j_i' 


then D . 
J 


D . , +1 . 
J-l 


Property A6--3 The codes obtained from Algorithm A6 exhibit 
the "nested" property. 

Theorem 6.1 w__( fgl } = D . = d .... for all i, where d_ 

H 1“ J j j FREE I 

is the free distance of the code with memory order j. 


FREE 


Proof w H ( [c[] ) = Dj £ ky property A6-1 and theorem 

2.4. d £ w u ( [a] ) by property F2 of the free distance. 

j 


FREE “ H 
J 

FREE 


Therefore d = w^( j'.gj } for all j.| 


Theorem 6 . 2 For all the codes obtained from Algorithms Al, 

A2, A3, A4, and A5, = ^ = d • 


m 


Proof d = d = w ( 
FD m H 


|^j2j ) is a property of the codes ob- 

m 

tained from Algorithms Al, A2, A3, A4, and A5, and W H ( [a] ) 

m 
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= 3p REE follows from property F2. | 

In general Algorithm A6 will result in generators with 
greater weight than those obtained from Algorithm Al. There- 
fore, < 3 free for the codes obtained from Algorithm A6 will be 
larger than d^ for the same length codes obtained from 
Algorithm Al. Clearly, it is wise to choose L as large as 
is computationally possible in Algorithm A6. 

Table 6.6 shows the results of applying Algorithm A6 to 
the construction of an R = 1/2 fixed canonic systematic 
binary code with m = 35 and L =71. The adjoint of this code 
has dp REE = 18 and is also given in Table 6.6. It is inter- 
esting to note that Algorithm Al produced a code with m = 35 
and <3p REE = dp D = 13. Algorithm A6 resulted in a code with 

m = 35 and = 17. dp D was checked for this code and 

found to be 13. Therefore, Algorithm A6 produced a code with 

the same length and the same d^, but with a larger dp REE . 

Although the two codes have the same d . the code obtained 

PD 

from Algorithm A6 should exhibit a lower probability of error 
when used with sequential decoding. This was verified by 
simulating a sequential decoder for use on a binary symmetric 
channel on the Uni vac 1107. The results of this simulation 
will be presented in Section VI. C. 

Note that all the codes constructed in Section VI. A have 
distances considerably better than the non-asymptotic Gilbert 
lower bound and are longer than any previously known good 
codes. Also, each code has the property of minimizing the 
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TABLE 6.6 

CODES OBTAINED FROM ALGORITHM A6 


^FREE 


0 

1 

2 

1 

1 

1 

3 

1 

2 

1 

4 

0 

3 

0 

4 

1 

4 

1 

5 

0 

5 

1 

6 

1 

6 

0 

6 

0 

7 

1 

7 

1 

8 

0 

7 

1 

9 

1 

8 

0 

10 

0 

8 

0 

11 

0 

8 

1 

12 

1 

9 

0 

13 

1 

10 

0 

14 

0 

10 

1 

15 

0 

10 

0 

16 

0 

10 

0 

17 

0 

10 

0 

18 

1 

11 

0 

19 

1 

12 

0 

20 

1 

13 

1 

21 

0 

13 

1 

22 

0 

13 

1 

23 

0 

13 

0 

24 

0 

13 

0 

25 

1 

3-4 

1 

26 

1 

15 

0 

27 

1 

16 

0 

28 

0 

16 

0 

29 

0 

16 

1 

30 

0 

16 

0 

31 

0 

16 

0 

32 

0 

16 

1 

33 

0 

16 

1 

34 

0 

16 

1 

35 

1 

17 

1 


= adjoint codes 


The adjoint code with m = 35 has <3 FREE = 18 « 
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number of modulo- 2 adders needed in the encoding circuit for 
codes of a given distance and constraint length. 

B. More Construction Algorithms for R = 1/2 Codes 

It was shown in theorem 6.2 that the codes produced 
by Algorithms Al through A5 have dp REE = d^. This is a 
direct result of minimizing the number of modulo- 2 adders 
needed in the encoding circuit. Hence this property must be 
abandoned if codes with dp^g considerably larger than dp D 
are to be obtained. In this section a number of algorithms 
are presented for producing fixed R = 1/2 binary codes with 
large free distance. 

1. Systematic R = 1/2 Codes 

Since a low density of l's in the generator necessarily 

produces a code with low dp REE , the following algorithm was 

designed to produce a high density of l's in the generator. 

( 2 ) 

(For convenience, let 9jj_ v ' — 9j») 

Algorithm A7 

(0) Set g Q = 1, d Q = 2, and j = 1. 

(1) Set g^ = 0. 

(2) Compute d . . If dj > d^ , go to (4). 

(3) Set g^ = 1 and compute d j . 

(4) If j = m, stop. Otherwise, set j = j + 1 
and go to (1) . 

This code and its adjoint are shown for m = 35 in Table 6.7 
along with the non-a symptoti c Gilbert lower bound. Note that 
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TABLE 6.7 

CODES OBTAINED FROM ALGORITHM A7 


j 

g j 

A 

s j 

d. 

J 

d 

G 

0 

1 

1 

2 

2 

1 

1 

1 

3 

3 

2 

1 

0 

3 

3 

3 

0 

1 

4 

4 

4 

1 

0 

4 

4 

5 

1 

1 

5 

4 

6 

1 

1 

5 

4 

7 

1 

1 

5 

5 

8 

0 

0 

6 

5 

9 

1 

0 

6 

5 

10 

1 

1 

6 

5 

11 

0 

1 

7 

6 

12 

1 

1 

7 

6 

13 

1 

1 

7 

6 

14 

1 

1 

8 

6 

15 

1 

1 

8 

7 

16 

1 

0 

8 

7 

17 

1 

0 

9 

7 

18 

1 

1 

9 

7 

19 

1 

0 

9 

8 

20 

1 

0 

9 

8 

21 

1 

0 

9 

8 

22 

1 

1 

9 

8 

23 

1 

0 

9 

9 

24 

0 

0 

10 

9 

25 

1 

1 

10 

9 

26 

1 

1 

10 

9 

27 

1 

1 

10 

9 

28 

0 

1 

11 

10 

29 

1 

0 

11 

10 

30 

1 

1 

11 

10 

31 

1 

0 

11 

10 

32 

1 

1 

12 

11 

33 

1 

0 

12 

11 

34 

0 

0 

13 

11 

35 

1 

1 

13 

11 


= adjoint codes 
d G = ^GILBERT BOUND 
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dj must be recomputed in step (3) since, unlike Algorithm 
Al where only the choice = 1 can increase dj , either g^ 


= 0 or g^ 


1 can increase d ^ . 


Since it is known that a randomly constructed code is 
with high probability a good code [31 ] , the following 
algorithm was designed to keep the number of l's and 0's in 


the generator sequence g 


01 


( 2 ) 


equal. (For convenience, let g^ 


*11 

( 2 ) 


( 2 ) 


*21 


( 2 ) 


> 1 
/ • • • 


about 


- V 


Algorithm A8 

(0) Set g Q = 1, d Q = 2, w = 0, and j = 1. 

(1) Set g. = 0. 

(2) Compute d.. If d. > d , go to (7). 

J 3 J-l 

(3) Set g^ = 1. 

( 4} Compute d . . If d . > d . . , set w = w + 2 
3 J J-l 

and go to (7) . 

(5) If j > w, set w = w + 2 and go to (7) . 

(6) Set g^ = 0. 

(7) If j = m, stop. Otherwise, set j = j + 1 
and go to (1) . 

This code and its adjoint are given for m = 35 in Table 6.8 

along with the nan-asymptotic Gilbert lower bound. Again 

note that either g . = 0 or g . = 1 can increase d.. 

J J J 

Algorithm A8 can be modified to provide merely an ex- 
tension of Bussgang's optimal codes from m = 15 to m = 35. 
Both of Bussgang's optimal m = 15 codes were extended using 
Algorithm A8 and the resulting codes, along with their ad- 
joints and the non-asymptotic Gilbert lower bound, are given 
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TABLE 6.8 

CODES OBTAINED PROM ALGORITHM A8 


j 

s j 

A 

g j 

d. 

J 

fl G 

0 

1 

1 

2 

2 

1 

1 

1 

3 

3 

2 

0 

1 

3 

3 

3 

1 

0 

4 

4 

4 

0 

1 

4 

4 

5 

1 

1 

5 

4 

6 

0 

0 

5 

4 

7 

1 

1 

5 

5 

8 

1 

0 

6 

5 

9 

0 

1 

6 

5 

10 

0 

0 

7 

5 

11 

1 

0 

7 

6 

12 

0 

1 

7 

6 

13 

1 

0 

8 

6 

14 

0 

1 

8 

6 

15 

1 

0 

8 

7 

16 

0 

1 

8 

7 

17 

1 

1 

8 

7 

18 

1 

1 

9 

7 

19 

0 

1 

9 

8 

20 

0 

1 

9 

8 

21 

1 

0 

9 

8 

22 

0 

0 

9 

8 

23 

0 

0 

10 

9 

24 

1 

0 

10 

9 

25 

1 

1 

10 

9 

26 

0 

1 

11 

9 

27 

1 

1 

11 

9 

28 

0 

1 

11 

10 

29 

1 

0 

11 

10 

30 

0 

o • 

11 

10 

31 

0 

1 

12 

10 

32 

1 

1 

12 

11 

33 

1 

0 

12 

11 

34 

0 

1 

13 

11 

35 

1 

0 

13 

11 


= adjoint codes 
= ^GILBERT BOUND 
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in Table 6.9. Finally d^ and <3p REE are given for each of 
the eight codes constructed in this section in Table 6.10. 
Again, dp D is the same for a code and its adjoint, but dp REE 
is not necessarily the same since the set of codeword weights 
are identical only over one constraint length. For all ex- 
cept the code of Algorithm A7, it is possible to give only 

a range for d_ . Note, however, that for each code d 

* FREE free 

is considerably larger than d^ and that d^ i s about the same 

as for the m = 35, r = 1/2 codes constructed in Section VI. A. 
Hence it is reasonable to expect that these codes should 
perform considerably better with sequential decoding than 
those of the previous section. That they do will be demon- 
strated in Section VI. C. 

2. A Non-systematic R = 1/2 Code 

It has been noted that non-systematic codes are capable 
of providing a lower probability of decoding error than 
systematic codes for sequential decoding. However it is also 
desirable that the "quick-look" and ease of implementation 
properties of systematic codes be retained. Massey [32] has 
shown that the class of fixed binary R = 1/2 non-systematic 
codes for which a Gilbert lower bound on dp was proved in 

Section IV. D has these properties. For these codes 

(D) = D + G.^ 1 * (D) or G (2) (D) = D + G (1) (D) . (143) 

Therefore, for any information sequence whose transform is 
x(D) , 
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TABLE 6.9 

EXTENSIONS OF BUSSGANG'S OPTIMAL M = 15 CODES 





BUSSGANG 

1 


BUSSGANG 

2 

j 

d G 

g j 

A 

g j 

d. 

J 

g j 

A 

g j 

d. 

J 

0 

2 

1 

i 


1 

1 


1 

3 

1 

i 


1 

1 


2 

3 

0 

i 


1 

0 


3 

4 

1 

0 


0 

1 


4 

4 

0 

i 


1 

0 


5 

4 

1 

i 


1 

1 


6 

4 

0 

0 


0 

0 


7 

5 

1 

1 


1 

1 


8 

5 

1 

0 


0 

1 


9 

5 

0 

1 


1 

0 


10 

5 

0 

0 


0 

0 


11 

6 

1 

0 


0 

1 


12 

6 

0 

1 


1 

0 


13 

6 

1 

0 


0 

1 


14 

6 

1 

0 


0 

1 


15 

7 

1 

0 

9 

0 

1 

9 

16 

7 

0 

0 

9 

1 

1 

9 

17 

7 

0 

0 

9 

1 

1 

9 

18 

7 

0 

1 

9 

0 

0 

9 

19 

8 

1 

1 

9 

0 

1 

10 

20 

8 

1 

1 

10 

1 

0 

10 

21 

8 

0 

1 

10 

1 

1 

10 

22 

8 

0 

1 

10 

0 

0 

10 

23 

9 

1 

1 

10 

1 

0 

10 

24 

9 

0 

1 

11 

0 

1 

10 

25 

9 

1 

0 

11 

0 

0 

11 

26 

9 

0 

1 

11 

1 

0 

11 

27 

9 

1 

0 

11 

1 

• 1 

11 

28 

10 

0 

0 

12 

1 

1 

12 

29 

10 

1 

1 

12 

0 

0 

12 

30 

10 

0 

1 

12 

0 

1 

12 

31 

10 

0 

1 

13 

1 

1 

12 

32 

11 

1 

1 

13 

0 

1 

12 

33 

11 

1 

0 

13 

0 

0 

13 

34 

11 

0 

0 

13 

1 

1 

13 

35 

11 

1 

1 

13 

1 

1 

13 


= adjoint codes 

^ 

G ~ GILBERT BOUND 
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TABLE 6.10 

DISTANCE PROPERTIES OF 8 m = 35 CODES 


^FD 


Algorithm A7 

13 




16 

A7 adjoint 

13 

18 

£ 

^FREE 

< 

22 

Algorithm A8 

13 

16 

< 

^FREE 

< 

20 

A8 adjoint 

13 

18 

£ 

^FREE 

< 

22 

A8-BUSSGANG 1 

13 

16 

< 

^FREE 

< 

20 

A8--B1 adjoint 

13 

18 

$ 

^FREE 

< 

20 

A8-BUSSGANG 2 

13 

18 

< 

^FREE 

< 

20 

A8-B2 adjoint 

13 

17 

< 

^REE 

< 

23 
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y (l) (D) + y (2> (D) = x(D) G (1> (D) + x(D) G (2) (D) = x(D) 

[g (1 > (D) + D 4 - G (1) (D)] = D x(D) (144) 

and the information sequence can be obtained from the two 

encoded sequences with a delay of one time unit simply by 

(IV (2V 

adding together y v ' (D) and y v ' (D> . This allows a quick 
look at the data sequence to be made before submitting the 
received sequence to error correction, i.e., this code has 
the "quick-look" property. (Clearly, the first K received 
sequences alone provide the "quick-look" for canonic system- 
atic codes.) 

Consider the following realizable functions 


V(D) = 1 + P m+1 
x(D) 1 + D 


2 m 

1 + D + D + . . . + D . 


(145) 


Form (145) it follows that 

y(D) + Dy (D) = x(D) + D m+1 x(D) (146) 

or 

y(D) = x(D> + D [y (D) + D m x(D)] . (147) 

A linear sequential circuit (LSC) which realizes equation 

— ( i) 

(147) is shown in Figure 6.4. If G (D) is the complement 
of G^ (D) , then 

(D) = x(D) G ( ^ (D) = x(D) [g ( j) (D) + 1 + D +...+D m ]. 

(148) 

A circuit which realizes equation (148) is shown in Figure 
6.5. Hence if G^ (D) has a high density of l's and therefore 
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requires many modulo-2 adders in its realization, G VJ; (D) , 
which requires few modulo-2 adders in its realization, can 
be implemented instead and the output complemented to produce 
(D) . This fact will be used to greatly reduce the com- 
plexity of the class of R = 1/2 codes with (d) = d + g^ 

(D) . A code of this type will now be constructed. (For con- 


venience, let g.. 

J 1 

Algorithm A9 


( 1 ) _ „ ( 1 ) 


and g_ 


( 2 ) 

9 j •> 


(0) Set g Q v = g =1, d x = 3, and j = 2. 

(1) set g^ 1 * 9 s 0. 

( 2) Compute d . . if d . > d . , / go to ( 4) . 

j j j -I 

(1> 

(3) Set g^ =1. 

(4) If j = m, stop. Otherwise, set j = j + 1 
and go to (1) . 

Note that the set of weights of the codewords |^yj with 

f *1 V /TV J 

x / 0 is the same whether g, ' = 0 or g, =1 since com- 
0 J 1 

piemen ting x^ cancels the effect of complementing g^ ' . 

Hence there is no need to recompute d. in step (3) as in 

( 1 ) 3 

Algorithm A7 since if setting g. =0 does not increase d., 

( 1 ) 3 3 

then neither does setting g^ =1. Also, an increase of 

two in d . at any step is clearly impossible, since if y. = 

«— __ J p* — J 


£l,lj for some input sequence jx] ' then y = |^0,oJ f° r the 
input sequence [x 1 ] __ = [[x] ._ ± , x . j , and the column 

distance does not increase at all. Hence at each step in the 
algorithm, d^ either increases by 1 or stays the same. The 
code produced by Algorithm A9 with m = 35 is given in Table 


for the 


6.11 along with the non-asymptotic Gilbert lower bound. 
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TABLE 6.11 

CODES OBTAINED FROM ALGORITHM A9 


j 


(2) 

g j 

d j 

d G 

0 

1 

1 

2 

2 

1 

1 

0 

3 

3 

2 

1 

1 

3 

3 

3 

0 

0 

4 

4 

4 

1 

1 

4 

4 

5 

1 

1 

4 

4 

6 

0 

0 

5 

4 

7 

1 

1 

5 

5 

8 

1 

1 

5 

5 

9 

1 

1 

5 

5 

10 

0 

0 

6 

5 

11 

1 

1 

6 

6 

12 

0 

0 

7 

6 

13 

1 

1 

7 

6 

14 

1 

1 

7 

6 

15 

0 

0 

8 

7 

16 

1 

1 

8 

7 

17 

1 

1 

8 

7 

18 

1 

1 

8 

7 

19 

1 

1 

8 

8 

20 

0 

0 

9 

8 

21 

1 

1 

9 

8 

22 

1 

1 

9 

8 

23 

1 

1 

9 

9 

24 

1 

1 

9 

9 

25 

1 

1 

9 

9 

26 

0 

0 

10 

9 

27 

1 

1 

10 

9 

28 

1 

1 

10 

10 

29 

1 

1 

10 

10 

30 

0 

0 

11 

10 

31 

0 

0 

11 

10 

32 

1 

1 

11 

11 

33 

1 

1 

11 

11 

34 

1 

1 

11 

11 

35 

1 

1 

11 

11 


d G ^GILBERT BOUND 
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The free distance for this code is known to be at least 
17, and is probably much higher. The complete encoding cir- 
cuit is shown in Figure 6.6. Note that only 11 modulo-2 
adders are needed in the encoding circuit. This is exactly 
the same number needed to implement the m = 35 code of 
Algorithm A1 which has dp REE = 13. Hence a substantial gain 
in (and therefore in decoding probability of error) has 

been achieved without sacrificing anything in encoder com- 
plexity or "quick-look" capability. A truly surprising re- 
sult I 

Most of the codes presented in this section could have 
been easily extended out to about m = 60. However m = 35 
seems to be a convenient length for many applications. 

C. Performance of Codes with Sequential Decoding 

1. Brief Description of the Simulated Sequential Decoder 

In order to test the codes constructed in this chapter 
along with other known good codes, a sequential decoder was 
simulated on the Univac 1107 at the University Computer Center. 
Two simulations were made, one for a BSC and one for a 
Gaussian channel. Each program consists of four parts: a 

main program DECODE for reading in data and printing out re- 
sults, a subprogram RANGEN for generating random noise, a sub- 
program TABSET for converting the random noise into tabular 
form suitable for the sequential decoder, and a subprogram 
SECO for the sequential decoding algorithm. Special thanks 
are due to Dr. K. Vairavan, who programmed both the RANGEN 
and TABSET subprograms, to Mr. John Geist and Mr. James Wruck 
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for their numerous contributions to the efficiency of the 
programs, and to Mr, J. Chang and Mr. John Brennan for the 
preparation of the Gaussian program. 

Each subprogram was written in assembly language to 
make the program as fast as possible, while the main program 
was written in FORTRAN to facilitate the input-output. Input 
information needed for the operation of the BSC program is 
as follows (for a complete discussion of sequential decoding 
parameters and notation, see Gallager 25 J }% 

(1) channel error probability p; 

(2) the memory m of the code; 

(3) the generator of the code being tested; 

(4) the threshold increment H of the se- 
quential decoding search; 

(5) a constant CONMET used to spread the 
difference between the metric values; 

(6) bins for the number of computations. 

R and the metric values are then computed from p 
comp 

and CONMET. The threshold increment H used in the production 
runs was determined experimentally. The value of H which 
optimizes the bound on computation is known to be 2 [ 3 ] . 
Since CONMET was chosen as 8, the "-optimum" H is 16. How- 
ever, through testing a single code for different values of 
H, it was determined that choosing H to be 32 was a better 
choice from both a computational and probability of error 
standpoint. These results are shown in Table 6.12. 



TABLE 6.12 

EFFECT OF VARYING THE THRESHOLD INCREMENT H 
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Code No. 1 Code Name 

Memory = 35 Rate = Typ 

Generator Sequences (Octal): 400000000000 

651102104421 

Known Distance Properties: 

d FD = 13 

Nature of Construction: Algorithm A1 


Code Name Minimum Weight Code 
Type Systematic 


Simulation Results: 

(1) Channel BSC: p = .033 


Total 

H 4 Frames loop 


Error 
Frames iq 


Erased 

Frames 


Computation: 


N 


# Frames 
with #C - N 


Total Error Bits: 34 


2921 310 350 


.000 10001000 


(2) Channel ^SC: P “ *0 33 


Computation: 


N 


# Frames 
with #C - N 


292 310 350 


1000 looqiooo 


292 310 350 


10001000 1000 ! 


(3) Channel BSC: p = .03 
Computation: 


N 


w Frames 
with #C - N 


(4) Channel BSC: p n ,035 
Computation: 


N 


# Frames 
with #C - N 


(5) Channel BSC: p = .033 
Computation: 


N 


# Frames 
with #C - N 


(6) Channel BSC: p = .033 
Computation: 


N 


§ Frames 
with #C - N 


400 475 1550 700 125 


1000 1000 999 986 607 


. Total . ... 
_ H Frames i000 


400 475 550 700 1250 


1000 991 957 756 226 


Total 

_ H 16 Frames 1000 


400 475 550 700 1250 


989 869 665 369 85 


Total 

H 32 Frames 1000 


2500 

'.5000 

10K 

20K 

5 OK 

154 

42 

15 

4 

0 


Error 

Frames 


Erased 

Frames 


Total Error Bits: 34 


5000 

10K 

20K 

50K 

16 

5 

0 

0 


Error 
Frames 11 


Total Error, .Bits : 37 


5000 

10K 

20K 

5 OK 

10 

0 

0 

0 


Error 
Frames 16 


Erased 

Frames 


Total Error Bits: 45 


292 

310 

350 

400 

475 

550 

700 

1250 

2500 

1000 

100 

1 

992 

880 

585 

397 

206 

53 

16 


10K 

20K 

5.0K 

1 

0 

0 


Total 

H 64 Frames 1000 


Error 
Frames 26 


Erased 

Frames 


Total Error Bits: 62 


292 

310 

350 

400 

475 

550 

700 

1000 

100C 

993 

886 

642 

502 

315 


10K 

20K 

50K 

3 

1 

0 


Total 

H 128 Frames 1000 


Error 

Frames 177 


Erased 

Frames 


Total Error Bits: 488 



Erased 
Frames 0 




292 

310 

350 

400 

475 

550 

700 1 

1250 

2500 

500 

1000 

100C 

1000 

997 

982 

954 

891 

565 

227 

86 


10K 

20K 

50K 

22 

9 

1 
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Each production run consisted of 1000 frames of 256 
branches (blocks of information digits) each for a particular 
code and a particular channel error probability p. A frame 
was cut off and considered to be "erased" if it reached 
50,000 computations. If a frame was decoded perfectly, it 
took (256 + m) computations since 256 information blocks gen- 
erate (256 + m) transmitted blocks and the algorithm would 
count one computation for each correctly decoded block. 

Hence the computational bins are just numbers inclusive be- 
tween (256 + m) and 50,000 which record how many frames 
reached or exceeded that number of computations for decoding. 
Usually 13 computational bins were chosen for each produc- 
tion run. 

E b 

In the Gaussian program the signal -to -noise ratio — 

0 

must be read in instead of p, where is the energy per 
information digit and Nq is the noise power spectral density. 
Then the procedure outlined in Jacobs [ 33 ] is followed to 
compute the metric values needed by the sequential decoder. 

Output information available from the BSC program in- 
cludes the followings 

(1) the actual branch metric values and R ; 

comp 

(2) for each decoded frames 

(a) the number of computations; 

(b) the number of decoding errors; 

(c) the last branch decoded if the frame 


is erased; 
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(d) the received sequence; 

(e) the decoded sequence; 

(3) for the entire 1000 decoded frames; 

(a) the number of erased frames; 

(b) the number of incorrectly decoded frames; 

(c) the number of correctly decoded frames; 

(d) the distribution of computation into 
bins. 

Clearly the total number of error digits can be easily cal- 
culated from (2b). When the number of computations reached 
50,000, decoding was terminated and the frame declared 
"erased". The output then recorded how far the search had 
progressed into the code tree when decoding was terminated. 
The printout of the received sequence and the decoded se- 
quence for each frame is optional in the program. 

For each computational bin, the number of frames which 
reached or exceeded that amount of computation is recorded. 
For example, the bin labeled 50,000 always contains the 
number of "erased" frames, and the bin labeled (256 + m) 
always contains the total number of frames. 

In the Gaussian program, additional output information 
about the channel is available. 

In the RANGEN subprogram, a library subroutine is used 
to generate a noise sequence distributed according to the 
channel error probability p for the BSC program. In the 
Gaussian program, the noise sequence is distributed accord- 
ing to the quantized channel model given by Jacobs 33 j . 
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TABSET merely converts the noise sequence into tabular data 
for use by SECO. 

SECO is the actual sequential decoding algorithm. The 

version used is thoroughly discussed by Gallager [ 25 ] . A 

flow chart for SECO is shown in Figure 6.7. It is always 

assumed that the all-zero sequence has been transmitted. 

Since this was known to the programmer, SECO was always 

biased to look out on a 1 branch before looking out on a 0 

branch in case the metric values on the two branches were 

1 

tied. (Here the discussion pertains only to R = - codes, 
in which there are only two branches emanating from each 
node in the code tree.) This undoubtedly resulted in slight- 
ly more computation than would be required normally, but of 
course this deficiency was common to all runs and would be 
expected to have no effect on the comparison between differ- 
ent codes. 

A computation was counted as a "forward look", i.e., 
every time the decoder looked forward on a branch, and at 
no other time, a single computation was counted. Each com- 
putation, including the calculation of the parity digits, 
took about 100 jasec of computer time. 

The SECO algorithm is capable of handling both system- 
atic and non- systematic codes with m < 72. Programs actually 
available are for R = 1/2, R = 1/3, and R = 1/4 only. How- 
ever, only results on R = 1/2 codes will be reported here, 
since they are sufficiently representative of all rates. 
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LP * look forward BB = best branch V - node value 

T = threshold H - threshold Increment 

MP = move forward LB * look back MB = move back 

WB = worst branch 

Pig. 6.7. SECO flow chart . 
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Also, data was taken for only three values of p and one 
E b 

value of ~. These values are very typical, though, of 
0 

a practical randomly distributed space channel. For p = 

.033, i.e., R = 1/2 = (0.9) R , each production run of 

comp 

1000 frames took about two minutes of computer time. For 


p = .045, i.e., R = 1/2 = R , each run took about four 

comp 

minutes. For p = .057, i.e., R = 1/2 = (1.1) R / each 

E]-, comp 

run took about 20 minutes. And for = 2 or 3 db, each 

0 

run took about five minutes. 


2. Comparative Analysis of Codes 

In Appendix A charts are given which have complete 
information on 13 different codes. A name and number is 


assigned to each code for identification purposes, and the 
means of construction for each code is briefly explained. 
Simulation results are given for the four channels des- 
cribed above. Not all the codes were tested with p = .057, 
since the computation time was so long. 

An interesting comparison can be drawn between code 1 
(from Algorithm Al) and code 3 (from Algorithm A6) . Note 
that there are fewer error frames for code 3. This appears 

to be due to the fact that d „„ is larger for code 3, 

FREE 

since d^ is the same for both codes, and substantiates 
the previous statement that cl is a more important param- 
eter than for sequential decoding. 

Also compare code 11 (the non- systematic code from 
Algorithm A9) with code 12 (from Forney [28] ), „ The non™ 
systematic code is clearly superior in number of error 
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frames, although it has more erased frames. For the noisi- 
est BSC, p = .057, code 11 makes no decoding errors while 
code 12 incorrectly decodes about 10 percent of the frames. 
However code 11 erases about 15 percent more frames than 
does code 12. But of these frames it appears that about 
half of them were incorrectly decoded by code 12. Massey 
[ 32 ] has termed this a "fools rush in where angels fear to 
tread" phenomenon. The slight computational advantage of 
code 12 over code 11 is clearly due to this phenomenon. 

Since code 11 is more easily implemented than code 12 and it 
has the "quick-look" property, the conclusion is that it is 
far superior to code 12 in system performance as well as 
system complexity. In fact, code 11 did not make a single 
decoding error in all four simulations. To the author's 
knowledge, code 12 is generally considered the best m = 35 
systematic code available for sequential decoding. The per- 
formance of code 11 verifies the earlier statement that 
better results can be obtained for non- systematic codes than 
for systematic codes when used with sequential decoding (since 
more free distance is available for non-systematic codes) . 

Finally, compare the performance of code 2 with code 1. 
This indicates the advantage of using longer codes. However, 
encoder complexity increases with code length, which is an 
important consideration in many applications. 



VII, Deriving Good Convolutional- Codes 
from Cyclic Codes 

For completeness, this chapter will summarize at- 
tempts to derive good convolutional codes from good block 
codes, or at least to bound the distance of a convolu- 
tional code from known distance bounds on block codes. 

Some of these attempts have been quite successful, such as 
the codes due to McEliece [lo] presented in Section VII. A, 
others relatively unsuccessful, such as those in Section 
VII. B, but still of some interest for their structure. In 
Section VII. D a possible new approach to this problem is 
discussed, and some limitations inherent in deriving con- 
volutional codes from block codes are given in Section VII. E. 
In Section VII. C a method is presented for converting a code 
described by a parity-check matrix into a rationally equiva- 
lent generator matrix which is then reduced to Forney's [6] 
canonic generator matrix. 

A. McEliece 1 s Codes 

The following result was obtained by McEliece [loj for 
fixed R = 1/2 systematic codes only. If g(x) is the gen- 
erator of a cyclic code with minimum distance d and h(x) is 

g 

the dual generator of g(x) with minimum distance d^, then 

( 2) 

the fixed R = 1/2 systematic code with G (D) = g(D) has 

■W * min [ 1 + v 2 + «h] • <149! 

Note that in order to assure a large free distance a 
cyclic code must be chosen which has a large minimum distance 
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and whose dual code also has large minimum distance. Hence 
very low rate cyclic codes are not a good choice, since 
their dual generators have very low distances. 

2 5 g g To 11 

Let g(x) = 1 + x + x + x + x a + x x + x x . + 


Example 7.1 
12 

x , the dual generator of the Golay code. Then d 


8 , 


h(x) = 1 + x 2 + x 4 + x^ + x 6 + x^° + x 11 , and d^ = 7. Hence 

the fixed R = 1/2 systematic code with G^ (D) = 1 + D 2 + D 5 
S 9 10 11 12 

+D+D+D + D + D has dp REE > 9. Since the weight 

of the generator is 9, dp REE =9.1 

_ „ „ . » , 4 5 9 10 ^ 11 13 

Example 7.2 Let g(x) = 1 + x + x + x + x + x + x 

14 16 17 18 20 21 22 23 

+ x + x + x + x + x + x + x + x , the gener- 

24 4 5 

ator of an R = — cyclic code, d = 11, h(x) = 1 + x + x + 

4 / CJ 

8 9 11 12 15 18 20 23 24 ^ a .. 

x+x+x + x + x + x + x + x + x , and d, = l^- 

( 2 ) 

Therefore the fixed R = 1/2 systematic code with G (D) = 
g(D) has dp REE > 12. But since all parity sequences p pro- 
duced by an information sequence x with x Q / 0 either have 
weight at least 11 or x has weight at least 12 and p has weight 
at least 2, the only possible weight 12 codeword is produced 
by the information sequence x = £l,0,0,...J . But this code- 
word has weight 16. Hence dp REE > 13. Also the only possible 
codewords of weight 13 are those with weight 2 information 
sequences. But it can easily be shown that no information 
sequences of weight 2 can produce a codeword of weight 13. 

Hence dp REE ^ 14. But if the sequence whose transform is h(D) 
is chosen as the information sequence, a codeword of weight 


14 is obtained. Therefore 


d F: 


REE 


= 14. 
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These codes are comparable to those constructed in 
Chapter VI, and it appears that long medium rate cyclic 
codes with good distance properties will produce good 
fixed R = 1/2 systematic convolutional codes for sequen- 
tial decoding. 

McEliece 1 s result will now be extended to the im- 
portant class of fixed R = 1/2 non-systematic codes and 
also to fixed R = 1/3 systematic codes. Let g(x) and h(x) 
be two relatively prime polynomials such that g(x) 
h(x) = x n -l. Clearly g(x) and h(x)' each generate a cyclic 
code of length n, and they are each other's dual genera- 
tors. Let d be the minimum distance of the cyclic code 

g 

generated by g(x) and let d^ be the minimum distance of the 
cyclic code generated by h(x) . 

Now consider the fixed R = 1/2 non-systematic convolu- 
tional code with (D) = g(D) and (D) = h(D) . Using 

the Euclidean division algorithm on an arbitrary informa- 
tion sequence x whose transform is x(D) implies that 

x(D) = h(D) q-j^D) + r^ (D) = g(D) q 2 (D) + r 2 (D). (150) 

Therefore 

y (1 * (D) = x(D> G (1) (D) = q^DMD 31 -!) + r ± (D) g(D) 

(151) 

y (2) (D) = x(D) G (2), (D) = q 2 (D) (D n -1) + r 2 (D) h(D) 

and, since the remainder cannot have higher weight than the 
dividend after division by D n -1 for any n. 
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w H [y (1) (D)] +w H [y (2) (D)] 2: w H [r x (D) g(D)] + 

w H [ r 2 (D) h(D)J , (152) 

where the Hamming weight of a polynomial is defined to be 
the number of non-zero terms in the polynomial. Then, since 
degree £r^(D)J < degree ^h(D)J and degree ^r 2 (D)J< degree 
j^g(D)J , r 1 (D) g(D) has weight at least d g , r 2 (D) h(D) has 
weight at least cL , and 


d FREE ~ d g + d h (153) 

unless r (d) =0, r„ (D) =0, or r (D) = r (D) =0. 

X ^ x z 

Assume r^(D) =0, r 2 (D) / 0. Then w H (D)J > 

and w H [y^ (D) J > 2 since all multiples of D n -1 must have 

weight at least 2 . Hence 


d FREE ~ 2 + * 


Assume r^(D) / 0, r 2 (D) = 0. Similarly 


(154) 


^REE ~ 2 + d c 


(155) 


Assume r 1 (D) = r 2 (D) =0. Then h(D) q^D) = g(D) q 2 (D) = 
x(D) . Since h(D) and g(D) are relatively prime, 

q 2 (D) 

q x (D) = g(D) = g(D) f (D) , (156) 

q 2 (D) = h(D) f (D) , (157) 


and 

y (1) (D) = f (D) g (D) (D n -1) 
y (2) (D) = f(D) h(D) (D n -1) . 


(158) 


Again using the Euclidean algorithm, let 
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f(D) = g(D) Q 1 (D) + (D) = h(D) Q 2 (D) + R 2 (D) . 

Then 


Y (1) (D) = Q 2 (D) (D 2n -1) + R 2 (D) g (D) (D n -1) 
Y (2) (D) = (D) (D 2n -1) + R X (D) h(D) (D n -l} / 


W H 1 

y (1 V 

> 2d since degree 

g 

[r 2 (D) g(D)J 

W H 

y <2) (»] 

> 2d^ since degree 

[r x (D) h(D)J 


implies that 

^FREE ^ 2 ^ d h + d g* 

unless Rj^(D) = 0, R 2 (D) =0/ or R-^(D) = R 2 (D) = 0. 

Assume R 2 (D) = 0, R 2 (D) t 0. Then 

W * 2d g + 2 - 
If R 1 (D) t 0, R 2 (D) =0, then 

^FREE - 2d h + 2 * 

And if R x (D) = R 2 (D) =0, then f(D) = g(D) (^(D) = h(D) 
Q 2 (D) . Since g(D) and h(D) are relatively prime, 

q 2 (d > 

q (D) = h(D) — = h(D) F (D) , 

“1 g(D) 

Q (D) = g(D) F (D) , 

and 

y (1> (D) = F (D) g(D) (D 2n -1) 
y (2) (D) = F(D) h(D) (D 2n -1) . 

Again applying the Euclidean algorithm, F(D) can be 


(159) 

(160) 

(161) 

(162) 

(163) 

(164) 

(165) 

(166) 

(167) 


written as 
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F(D) = h(D) qj(D) + r^(D) = g(D) q£ (D) + r^(D) , (168) 


and 


y (1) (D) = qj(D)(D n -l ) 3 + r £(D) g(D) (D^-l) 
Y ( 2 ) (D) = q^(D)(D n -l ) 3 + r ^(D) h(D)(D 2 n -l) . 


(169) 


n 3 3n 

Since (D -l) / D -1, it is not immediately clear how to 

bound cl in this case. However, as is shown in Appendix 
FREE 

B, if 


y (1> (D) = Q 1 (D) (D n -l) k + R 1 (D) g(D) (D n -1) '* C ” 1 
y (2) (D) = Q 2 (D) (D n -l) k + R 2 (D) h(D) (D 11 -!) 1 *- 1 


(170) 


for any positive integer k, then 

[y (1> (D)] £ Ld and w R [y (2 > (d)] > Ld^ 


H 


where L = w. 


H 


(D n -1) 


g 

k-1 


(171) 


This proves the following result. 
Theorem 7.1 For the class of fixed R = 1/2 non-sy sterna tic 
codes defined above, dp REE ^ 2 + min £d^, • 1 


Note that there is very little improvement over the R = 1/2 
systematic codes. 

Now consider the fixed R = 1/3 systematic convolutional 
code with G* 1 * (D) = 1, (D) = g(D) , and (D) = h(D) . 

Following the same line of proof used to derive theorem 7.1 
and again employing the results of Appendix B, it is easy to 
arrive at the following result. 


Theorem 7.2 For the class of fixed R = 1/3 systematic con- 
volutional codes defined above, dp REE > 2 + min ^d^ + -1 , 

2d h' 2d g] - 1 
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Example 7 . 3 Again consider the Golay code and let 
G (1) (D) = 1, G (2) (D) = 1 + D 2 + D 4 + D 5 + D 6 + D 10 + D 11 , and 
(D) = 1 + D 2 + D 5 + D 8 + D 9 + D 10 + D 11 + D 12 . Then 


S’REE ~ 16 ' 


The extension of these results to other rates appears to 
be unrewardingly tedious. Furthermore, R = 1/2 and R = 1/3 
codes are of the most practical interest. Equation (149) and 
theorem 7 . 2 guarantee that good long fixed R = 1/2 and R = 1/3 
systematic codes can be found for use with sequential decod- 
ing. The result of theorem 7.1 for fixed R = 1/2 non-system- 
atic codes, however, is somewhat disappointing. 

B. Wyner-Sullivan Codes 

A class of very high rate fixed codes with d^ = 5, 
based on the structure of BCH codes, was discovered by Wyner 
[34] . Sullivan [ 35 ] has introduced simple encoding and al- 
gebraic decoding procedures for these codes and has shown 
that they do not exhinit the error propagation effect [36] 
characteristic of certain convolutional codes. 

Consider a parity check matrix of the following form: 


N-l 


a 

, 3. N-l 
(a ) 

0 

0 

0 


N-2 


a 

, 3., N-2 
(a ) 

0 

0 

0 


a 

3 

a 

0 

0 

0 


1 

1 

0 

0 

0 


0 0 

N-l N-2 
a a 

, 3. N-l, 3. N-2 
(a ) (a ) 

0 0 

0 0 

0 0 


0 0 

a 1 

a 3 1 ••• 

0 0 

0 0 

0 0 


t 


(172) 
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3T 

where a is a primitive element of GF(2 )„ Since each power 
of a can be expressed as a binary r-tuple, each successive 
block of parity check digits is shifted down r rows* Hence 

IT 

there are r parity checks per time unit and 2 - 1 distinct 

2 r -r-l 

non-zero powers of a. Therefore N-K = r and R = . 

2-1 

The convolutional code defined by an H matrix is taken 

to be precisely the set of sequences in the null space of 
T 

H , and no other sequences. Wyner showed that dp D = 5 for 
all the convolutional codes in this class by noting that the 
first 2r rows of (172) are also a parity check matrix for a 
double error correcting BCH code. ( Wyner and Ash [ 37 ] have 
defined dp D as the minimum number of columns of H which can 
add to zero, including at least one column from the first 
block. ) 

The decoder looks at three blocks of syndrome digits before 
estimating a block of error digits. Hence the decoding mem- 
ory m = 2 and the decoding constraint length n = 3N = 3 

A 

3T 

(2 -1) . Clearly as r -► R -*• 1 and in the limit infinite 
constraint length R = 1 codes with d^ = 5 are approached. 
Sullivan [35] has shown that d^ for all these codes lies 
well above the non-asymptotic Gilbert lower bound on <3p D « 

The chief disadvantage of the Wyner-Sullivan codes is 
that no method has yet been found to extend the results to 
other than m = 2, d^ = 5 codes. Clearly this is a serious 
limitation on the usefulness of the codes. 
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C. Forney Canonic Form 

Forney [6] has outlined an interesting procedure for 
converting an encoder into what he calls a minimal canonic 
encoder . which is rationally equivalent to the original en 


H- = 
“m 


coder. An example will be given for the Wyner-Sullivan code 
with r = 3 which is described by the following parity check 
matrix: 

1110 10 0 
0 1110 10 
110 10 0 1 

10111001110100 
11100100111010 
00101111101001 
000000010111001110100 
0000000111001 0 0111010 
000000000101111101001 


(173) 


4 _ _ 

This is an R = y code with d^ = 5, m = 2, and n^ = 21. 

The first step is to write the parity-check matrix H 
as an (N-K) x N matrix of parity-check functions H(D) . This 
results in the following matrix: 


T 

H (D) = 


1+D 

D 

1 

1 

1+D 

1 

1+D 

1+D 

D 

D 

1 

1 

1+D 

0 

D 

0 

1+D 

D 

0 

0 

1+D 


(174) 
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Then convert H (D) into canonic systematic form by performing 

T 

elementary column operations on H (D) which do not change 
its null space. This results in the following matrix: 


T 

H (D) = 


1 

D/l+D 

1/1+D 2 

1/1+D 

1 

2 

1+D+D /1+D 

1 

1 

D/l+D 

D/l+D 

1/1+D 

1 


(175) 


10 0 
0 10 
0 0 1 


Now H(D) can 


G(D) , as detailed 


G(D) 


10 0 0 
0 10 0 
0 0 10 
0 0 0 1 


be converted to the generator matrix 
in Wyner [38] , where 


1 

D/l+D 

1/1+D 2 

1/1+D 

1 

i+d+d 2 /i+d 

1 

1 

D/l+D 

D/l+D 

1/1+D 

1 


(176) 


Multiplying each row of G(D) by the least common multiple 
of its denominator terms results in the following matrix: 



1+D 2 

0 

0 

0 

1+D 2 

D+D 2 

1 



0 

1+D 2 

0 

0 

1+D 

1+D 2 

1+D+D 2 


G(D) = 

0 

0 

1+D 

0 

1+D 

1+D 

D 

(177) 


_ 0 

0 

0 

1+D 

D 

1 

1+D 



In Forney's [6] procedure, the greatest common denomina 
tor of the determinants of all the K x K submatrices of G (D) 
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2 

is calculated. This is found to be (1+D) . Then a ration- 
ally equivalent matrix which is a basis for the row space 
of the above matrix can be constructed. This leads to the 
following matrix: 

1+D 1+D 0 0 D 1 D 

0 1+D 1 0 0 D 1+D 

(178) 

0 0 1+D 0 1+D 1+D 0 

0 0 0 1+D D 1 1+D _ 

Finally G(D) is converted to what Forney [6] calls the mini- 
mal canonic matrix which is rationally equivalent to each 
of the above matrices. This results in the following matrix: 
1+D 0 1 0 D 1+D 1 

0 1+D 10 0 D 1+D 

0 0 1+D 0 1+D 1+D D 

0 0 0 1+D D 1 1+D 

Note that Forney's canonic form does not necessarily mean 
the code is in systematic form. 

Hence the Wyner -Sullivan code with r = 3 has a ration- 
ally equivalent encoder with encoding memory m = 1 and en- 
coding constraint length n A = 14. Therefore the encoding 
circuit for the code can be implemented with only one-stage 
shift registers. 
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D. Codes Dependent on the Length of Information 
Sequence Bound 

In Section VII. A, it was shown that the free distance 
of certain convolutional codes depends on the cyclic code 
properties of the polynomials in the matrix G(D) . In this 
section a different view of this relationship will be taken. 
Consider an R = 1/2 systematic fixed code with G = , 

where 3^ is a semi-infinite identity matrix and 



" (2) (2) 

(2) 

„ (2) 




g o g i 

g 2 

... g 
m 




„ (2) 

„ (2) 

„ (2) 

(2) 



g o 

g l 

... a . 
3 m-l 

% 


n — 


_ (2) 

(2) 

(2) 

(2) 

u - 


g o 

* * * g m-2 

g m-l 

g 

^m 




(2) 

(2) 

(2) 




g 0 

g l 

g 2 








(180) 


Note that (180) truncated after k rows could also serve 

as the generator matrix of an (n,k) cyclic code. Therefore 

if the length of the information sequence needed to produce 

the shortest minimum free weight codeword is known to be 

less than or equal to k, d^, D of the cyclic code provides a 

lower bound on <3p REE of the convolutional code. Hence if 

the length of information sequence bound were known to be m 

for R = 1/2 fixed systematic codes (cf . Chapter V) , d^ for 

all cyclic codes with generator g(x) and R > 1/2 would 

provide a lower bound on <3 j, ree for the fixed R = 1/2 system- 

( 2) 

atic convolutional code with G (D) = g(D) . This would 
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mean that the BCH bound on minimum distance [ 39 j could be 
applied to convolutional codes as well as to block codes. 
But, of course, this conjecture has yet to be proved. 

E. Some Upper Bounds on Free Distance 

Besides providing lower bounds on the distance pro- 
perties of certain convolutional codes, cyclic codes also 
provide upper bounds on distance for certain convolutional 
codes. 

Theorem 7 . 3 Given an R = 1/2 systematic fixed code with 
G^ (D) = G(D) = 1 + g^D + ... + D m and w H |^G(D)J > 2, then 

(1> a FD £ 1 + W H [ H(D> ] • 

(2 > SrEE * 2 + W H M ' 

where H(D) is the lowest degree polynomial such that H(D) 
G(D) = D n - 1. 

Proof 


(1) Let h be the information sequence whose transform 


is H(D) . Clearly h Q - 1 and w H ( [ h G ) = 1 + 


WL/h) • Therefore d 
H FD 


< 1 + WL . 


(2) w H (k G) = 2 + w H (h) . Therefore < 2 + w H 

[h(D)] . | 

Theorem 7.3 indicates that generators of low rate cyclic 

( 2 ) 

codes make poor choices of G (D) for R = 1/2 systematic 


fixed codes. However, since the maximal length polynomials 
are "pseudo random", it might be inferred that they would 
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/ 

make very good choices for g' ; (d) 'because Shannon [ 31 ] 
showed that a randomly constructed code has a very high 
probability of being a good code. Example 7.4 will prove 
this conjecture to be incorrect. However the distance pro- 
perties of the other "pseudo random" generator sequences 
are still open to investigation. 

Example 7.4 Consider the class of R = 1/2 fixed systematic 
( 2 )' 

codes with G (D) = g(D) , where g(x) is the generator of a 
maximal length code, given in Table 7.1. 

TABLE 7.1 


Rate of the Maximal 
Lenath Code 

Memory of the 
Convolutional Code 

Upper Bound on 
the Free Distance 

2/3 

1 

5 

3/7 

4 

6 

4/15 

11 

7 

5/31 

26 

8 

6/63 

57 

9 

7/127 

120 

10 


Clearly, the convolutional codes get very bad as the rate 
of the maximal length codes decreases.] 

Now a theorem will be given whose proof is similar to 
the proof of theorem 7.3(2), and hence will be omitted. 
Theorem 7.4 Given an R = 1/2 systematic fixed code with 


G^) (d) - g(D) + L (D) , where G(D) is a polynomial and 
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L (D) is a very low weight polynomial, then <3p REE < 2 + 
j w H + 1 | |^H(D)J , where H(D) is the lowest de- 

gree polynomial such that G(D) H(D) = D n - 1. | 

Theorem 7 . 4 provides a tight bound on <3 j. ree for those codes 
whose generator is very close to the generator of a code 
tightly bounded by theorem 7.3. 

The results of theorem 7.3 and theorem 7.4 can be ex- 
tended to fixed systematic codes of all rates and to fixed 
non- systematic codes, but the bounds are the tightest in 
the R = 1/2 systematic case. 



VIII. Summary, Conclusions, and Recommendations 
for Further Research 

In Chapter I, the formalism for convolutional encod- 
ing was introduced and various problems associated with 
convolutional codes were discussed. Two different nota- 
tional systems were presented, the D-transform approach 
introduced by Massey [ 4 ] and extended by Forney [6] , which 
is used only for fixed codes, and an extension of the vec- 
tor notation of Wozencraft and Reiffen [ 3 ] , which is used 
for both fixed and time-varying codes. 

From Chapter II onward, only binary codes, i.e., codes 
defined over GF(2>, were considered, primarily for conven- 
ience since many of the results are easily extended to non- 
binary codes. In Chapter II, feedback decoding minimum 
distance, definite decoding minimum distance, free distance, 
and reverse distance were defined for both fixed and period- 
ic codes. Also two new distance measures, column distance 
and row distance were introduced. It was claimed that <3p REE 
is a more appropriate distance measure for sequential de- 
coding than a fact verified in later chapters. A gen- 

eral definition of definite decoding minimum distance, valid 
for systematic and non- systematic codes of all rates, is 
still being sought. In Chapter IV, it was necessary to give 
an alternate definition of n^ and d^ in order to obtain a 
lower bound on d^ for non-systematic periodic codes. 
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In Chapter III, a new definition of encoder equiva- 
lence, called causal equivalence, was given which not only 
guarantees that the encoders produce the same output se- 
quences, but that they have the same column distance proper- 
ties when only causal information sequences are allowed. 

This is slightly different from Forney's [6] definition, 
termed rational equivalence here, where two equivalent en- 
coders can have different values of column distance. Then 
a method was given whereby a non-sy sterna tic encoder with no 
causally equivalent systematic encoder can be simply con- 
verted to a systematic encoder without decreasing column 
distance. Such an encoder is called causally dominant to 
the original encoder. Hence systematic encoders are always 
as good as non- systematic encoders as far as distance pro- 
perties are concerned. However, if a non-systematic poly- 
nomial encoder is converted to a causally dominant or a 
causally equivalent systematic encoder with rational func- 
tions and then each generator function is truncated to pre- 
serve the encoding memory, d is preserved but is 

r D r REE 

not. Finally, general parity-check matrices and syndrome 
forming circuits were obtained for non-systematic encoders 
of all rates. 

In Chapter IV, bounds were obtained on various distance 
measures. A lower bound on cL was shown for both systema- 
tic and non-systematic periodic codes. It is conjectured 
that these bounds hold also for fixed codes, but this remains 
unproved. Also a Gilbert lower bound on d_ was given for a 
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simply implemented subclass of periodic codes with period 
T = 2m + 1. A strong lower bound on dp REE was obtained for 
non-systematic periodic codes, and the result was used to 
prove an upper bound on error probability for maximum like- 
lihood decoding over a BSC which is superior to viterbi ' s [ 2 ] 
upper bound for low rates and which meets Viterbi 1 s lower 
bound at R = 0. An upper bound on clp REE for all fixed codes 
which is essentially the same as McEliece and Rumsey's [loj 
bound for fixed R = systematic codes was also shown. It 
was indicated how this result can also be extended to per- 
iodic codes. Finally, a Gilbert lower bound on a simply 
implemented subclass of fixed R = 1/2 non-systematic codes 
was obtained. However, a complete set of bounds on the 
distance properties of convolutional codes, most noticeably 
a Hamming upper bound on <3^, , is still missing. 

A major consequence of the results of Chapter IV was 
the demonstration that more free distance is available from 
non-systematic codes than from systematic codes of the same 
length. Hence it was conjectured that non-systematic codes 
would perform better with sequential decoding than systema- 
tic codes of the same constraint length, a fact verified 
experimentally as reported in Chapter VI. 

In Chapter V, some partial results were presented on 
the still outstanding problem of bounding the length of the 
information sequence needed to produce the minimum free 
weight codeword. Also some methods of calculating the free 
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distance were given, a problem which would be greatly simpli- 
fied by obtaining a tight length of information sequence 
bound. 

In Chapter VI, simple and efficient algorithms were 
given for constructing fixed systematic convolutional codes 
with dp D considerably larger than the non-asymptotic Gilbert 
lower bound out to m = 71 for R = 1/2, and m = 35 for R = 1/3 
and R = 1/4. The algorithms always retained the property 
of minimizing the number of modulo-2 adders needed in the 
obvious encoding circuit for codes of a given length and 
minimum distance, an important consideration in many appli- 
cations. In addition, an algorithm for constructing fixed 
systematic R = 1/2 codes with known d^, was presented. 

Other algorithms for constructing R = 1/2 fixed codes 
which do not minimize the number of modulo -2 adders in the 
encoding circuit were also given. In particular a construc- 
tion algorithm was presented for the class of fixed R = 1/2 
non- systematic codes for which a Gilbert lower bound was 
obtained in Chapter IV. These codes have been shown by 
Massey [32] to possess the "quick-look" property of systema- 
tic codes, and to have a very simple encoding circuit. The 
code of this type which was constructed was shown to have 
a very large dp REE , a consequence of its being a non- 
systematic code. A very strong result would be obtained if 
any of the algorithms presented in Chapter VI could be shown 


to produce good codes of arbitrary length. 
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Finally, a brief description of the simulated sequen- 
tial decoding program was given, and a comparative analysis 
of sequential decoding performance for various codes with 

memory 35 was presented. Four channels, a Gaussian channel 
E 

b 

with N =2.0 and three BSC's with R = 1/2 = 0.9 R 

0 comp 

R = 1/2 = R , and R = 1/2 = 1.1 R , respectively, were 

simulated. The fixed R = 1/2 non- systematic code was far 

superior to all the other codes tested as regards system 

performance. Indeed it did not make a single decoding error 

over any of the four simulated channels. This was due, of 

course, to the advantage in free distance which non-systema- 

tic codes possess over systematic codes. Also, because of 

its simple implementation, it was very desirable as far as 

system complexity is concerned. 

In Chapter VII, McEliece's [lo] result obtaining fixed 

R = 1/2 systematic codes with large d from cyclic codes 

FREE 

was extended to fixed R = 1/3 systematic codes and fixed 
R = 1/2 non-systematic codes. The results for the latter 
case were disappointing, but good fixed R = 1/3 systematic 
codes were found. Some codes discovered by Wyner [34] and 
Sullivan [35] were presented via the parity-check matrix, and 
a method was given for converting a parity-check matrix in 
non-systematic form into a rationally equivalent generator 
matrix. However the codes considered have not been gener- 
alized beyond m = 2 and d^ =5. A new approach to con- 
structing good convolutional codes from cyclic codes was 
introduced, but it was seen to depend on proving a tight 
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length of information sequence bound. Finally, some limita 
tions on constructing convolutional codes from block codes 
were presented. The problem of finding some construction 
technique which yields good codes as the memory gets arbi- 
trarily large remains unsolved for convolutional codes, as 
indeed it is still unsolved for block codes. 
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APPENDIX A 
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Code No. 

Memory = 35 


Code Name Minimum Weight Code 
\ Type Systematic 


Generator Sequences (Octal): 400000000000 

651102104421 


Known Distance Properties: 

d FD = 13 


d FREE 13 


Nature of Construction: Algorithm A1 


292 

400 

450 

500 

600 

700 

850 

100 

1000 

969 

906 

822 

681 

552 

426 

352 


Simulation Results: , 

(1) Channel Gauss: E l> 0~ * H 


Computation: 


N 


# Frames 
with #C - N 


(2) Channel BSC: p - .033 
Computation: 


N 


# Frames 
with #C - N 


( 3 ) Channel BSC: p - .045 
Computation: 


N 


# Frames 
with #C - N 


(4) Channel BSC: p o .057 


Computation: 


N 


# Frames 
with #C - N 


Total 

32 Frames 1000 


Error 

Frames 16 


Erased 

Frames 


Total . 
Frames ^000 


Total Error Bits: 91 


Erased 

Frames 


1200 

1500 

275 

203 


Error 

Frames 16 Frame 

Total Error Bits: 45 


292 

310 

350 

400 

4,75 

550 

700 

1250 

2500 

5000 

10K 

20K 

L000 

1000 

992 

880 

585 

397 

206 

53 

16 

4 

1 

0 



Total 

32 Frames 1000 


Error 
Frames 73 


Erased 

Frames 


292 

310 

350 

400 

475 

550 

700 

1250 

2500 

L000 

1000 

999 

991 

900 

769 

551 

261 

110 


Total 

32 Frames 1000 


Total Error.Bits: 355 


Erased 

Frames 


Error 

Frames ^28 


Total Error Bits: 6406 


400 

550 

700 

850 

1000 

1000 

938 

846 

775 

719 


(5) Channel 
Computation: 


N 


# Frames 
with #C - N 


(6) Channel 
Computation: 


N 


# Frames 
with #C - N 


Total 

Frames 




Error 

Frames 


5000j 

10K 

20K 

o 

LO 

287 

150 

56 

9 



Erased 

Frames 


Total Error Bits: 


Total 

Frames 


Error 

Frames 


Erased 

Frames 


Total Error Bits: 






































































































175 


Code No. 2 Code Name Minimum Weight Code 

Memory = 71 Rate = \ Type Systematic 

Generator Sequences (Octal) : 400000000000000000000000 

651102104421022041101101 

Known Distance Properties: 

d =21 d =21 

FD FREE 

Nature of Construction: Algorithm A1 


Simulation Results: „ , =90 Total Error Erased 

(1) Channel Gauss: b / o H 32 Frames 1000 Frames 9 Frames 4 


Computation: Total Error Bits: 26 

- - - — *r — ir t * - — — s' — —nr*" — i — - — t- — ■ — a'" nr— g g 


N 

292 

400 

450 

500 

600 

700 

850 

1000 

120C 

1500 

400C 

10K 

25K 



# Frames 
with #C - N 

1000 

995 

970 

904 

727 

607 

455 

372 

282 

212 

57 

13 

5 

■ 

■ 


Total Error Erased 

(2) Channel BSC: p = .033 H 32 Frames 1000 Frames H Frames 0 


Computation: Total Error Bits: 13 

' * '■ 1 > . | — a — a — — » f 1— ■ " ... , 


N 

292 

W0 

550 

700 

,850 

1000 

150C 

2000 

2500 

5000 

10K 

20K 

5 OK 


Hi 

§ Frames 
with #C - N 

1000 

978 

507 

249 

144 

81 

37 

23 

16 

5 

1 

0 

0 

■ 

■ 


Total Error Erased 

(3) Channel BSC: p = .045 H 32 Frames 1000 Frames 62 Frames 1 


Computation: ........ Total ErrorgBitsj^iQ 


N 

292 

400 

550 

700 

850 

1000 

1500 

200C 

2500 

500C 

10K 

20K 

5 OK 


HE 

# Frames 
with #C - N 

1000 

1000 

832 

589 

458 

363 

217 

146 

103 

29 

11 

5 

1 

■ 

■ 


Total Error Erased 

(4) Channel H Frames Frames Frames 


Computation: Total Error Bits: 

— i t. ,■■■■ — — — - "t — i a 1 a — —a t • — t - ■■ ■ § ■ — ■' ■'■ ""- a — — i” * 


N 















in 

# Frames 
with #C - N 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 


Total Error Erased 

(5) Channel H Frames Frames Frames 


Computation: Total Error Bits: 

* " i - — - b a 1" 1 a 1 ■ - a a — ■■■ ■■■ — ■ r •~r m t— ■■ 


N 
















# Frames 
with #C - N 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 


Total Error Erased 

(6) Channel H Frames Frames Frames 


Computation: Er:i ; 0:r Bits: 


N 
















# Frames 
with #C - N 



■ 

■ 
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Code No . __ 3 Code Name Minimum Free Weight Code 

Memory = 35 Rate = 3g Type Systematic 

Generator Sequences (Octal): 400000000000 

732460703401 

Known Distance Properties: 

d FD = 13 d FREE = 17 

Nature of Construction: Algorithm A6 


Simulation Results: Total Error Erased 

(1) Channel Gauss: b /1N o H 32 Frames loop Frames 5 Frames ^ 


Computation: Total Error Bits: 27 


N 

292 

400 

450 

500 

600 

700 

850 

1000 


1500 

IB 

10K 

25K 


zn 

# Frames 
with #C - N 

1000 

973 

900 

800 

651 

521 

407 

336 

264 

210 

60 

20 

8 

PH 

■ 


Total Error Erased 

(2) Channel BSC: p = .033 H 32 Frames 1000 Frames 1 Frames 0 


Computation: Total Error Bits: 5 


N 

292 

400 

550 

700 

850 


1500 

2000 

2500 

5000 

10K 

20K 

50K 



'# Frames 
with #C - N 

1000 

873 

379 

183 

121 

84 

33 

21 

14 

8 

2 

0 

0 

■ 

■ 


Total Error Erased 

(3) Channel BSC: p a .045 H 32 Frames 1000 Frames 17 Frames 3 


Computation: Total Error a Bits: 174 


N 

292 

400 

550 

700 

850 

1000 

EB 

O 

O 

O 

CM 

2500 

5000 

10K 

2 OK 

5 OK 


“1 

# Frames 
with #C - N 

1000 

992 

746 

522 

400 

327 

196 

147 

107 

45 

23 

13 

3 

■ 

■ 


(4) Channel BSC: P = *057 


H 32 


Total 

Frames 


1000 


Error 

Frames 


234 


Erased 
Frames 33 


Computation: Total Error Bits: 5226 

b. -■ — - ..... — — — f" i a -— — i - v -- “8 1 r i 11 ■ 1 ■ — x a "w — : 9 — . ... — 


N 

292 

400 

550 

700 

850 

1000 

1500 

2000 

EETiTi 1 

sood 

10K 

20K 

5 OK 


! 

# Frames 
with #C - N 

1000 

1000 

927 

823 

746 

683 

547 

481 

437 

309 

202 

107 

33 

■ 

■ 


Total Error Erased 

(5) Channel H Frames Frames Frames 


Computation: Total Error Bits: 

-fc — — 3 a-"-" - ■■■’ — m— a u ■■ ‘ s ■ 3 — 1 ■■■' 1 i 1 " — s — J ■ ■ a r~ 


N 















JKM 

# Frames 
with #C - N 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

|| 

m 


Total Error Erased 

(6) Channel H Frames Frames Frames 


Computation: . Total Error Bits: 


N 















zn 

# Frames 
with #C - N 

|| 

■ 

■ 

■ 

■ 

S| 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 
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Code No. 


Memory = 35 

Generator Sequences (Octal): 400000000000 

653110162117 


Code Name Minimum Free Weight Adjoint 
Systematic 


Rate = 3§ Type 


Known Distance Properties: 

d FD = 13 


d FREE 18 


Nature of Construction: Adjoint of code' no. 3 


Simulation Results: „ , 

(1) Channel Gauss: b' o~ H 

Computation : 


Total 

32 Frames 1000 


Error 
Frames 3 


Erased 

Frames 


Total Error Bits: 25 


N 

292 

400 

450 

500 

600 

700 

850 

1000 

1200 

1500 

400C 

10K 

25K 


zzi 

# Frames 
with #C - N 

100C 

972 

904 

815 

668 

537 

407 

Ppj 

263 

194 

66 

22 

10 

■ 

■ 


(2) Channel BSC: p = .033 
Computation 


Total 

32 Frames 1000 


Error 

Frames 


Erased 

Frames 


Total Error Bits: 0 


N 

292 

400 

550 

700 

850 

1000 

1500 

2000 

2500 

5000. 

10K 

WL 

5QK 


ZZ] 

§ Frames 
with #C - N 

1000 

880 

378 

190 

116 

83 

36 

18 

13 

7 

2 

0 

0 

■ 

■ 


(3) Channel BSC: p = .045 
Computation 


H 


Total 

32 Frames jnoo 


Error 

Frames 


_e_ 


Erased 

Frames 


Total Error*Bits:65 


N 

292 

400 

550 

700 

850 

1000 

1500 

EEEEi 

2500 

5000 

10K 

20K 

50K 


zzi 

# Frames 
with #C - N 

1000 

991 

744 

515 

410 

337 

207 

141 

116 

58 

23 

14 

n 

■ 

■ 


(4) Channel 
Computation: 


H 


Total 

Frames 


Error 

Frames 


Erased 

Frames 


Total Error Bits: 


N 









— 

pppp 





zz 

# Frames 
with #C - N 

■ 

■ 

■ 

■ 

■ 

■ 

|| 

■ 

■ 

mi 

■ 

PH 

Hi 

■ 

■ 


( 5 ) Channel 
Computation: 


H 


Total 

Frames 


Error 

Frames 


Erased 

Frames 


Total Error Bits: 


N 















ZZ 

# Frames 
with #C ? N 

■ 

■ 

■ 

|| 

■ 

■ 

H 

|| 

■ 

■ 

■ 

■ 

■ 

■ 

■ 


(6) Channel 
Computation: 


Total 

Frames 


Error 

Frames 


Erased 

Frames 


Total Error Bits: 


N 




KH 

M 

pMp| 

PPPPI 

pppp 

pppp 

■ ■ 

PPPP 

P| 



~Z 

# Frames 
with #C - N 

■ 

PP 


■ 

■ 

■ 

■ 

PP 

■ 

■ 

PP 

■ 

ii 

Hi 

Hi 
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Code No._ 
Memory = 


35 


Code Name Maximum Weight Code 
Rate = % Type Systematic 


Generator Sequences (Octal): 400000000000 

736677773575 

Known Distance Properties: 


FD 


13 


FREE 


16 


Nature of Construction: Algorithm A7 


Simulation Results: g 
(1) Channel Gauss: t> o 


0 Total 

H 32 Frames 1000 


Error 
Frames 3 


Erased 

Frames 


N 

292 

Elifil 

450 

500 

600 

700 

850 

1000 

1200 

1500 


10K 

25K 



# Frames 
with #C - N 

222 

974 

902 

fl 

652 

539 

414 

344 

278 

202 

65 

26 

8 

B 

a 


(2) Channel BSC: p = .033 


Total 

H 32 Frames 1000 


Error 

Frames 


0 


Erased 

Frames 


Total Error Bits: 0 


N 

292 

400 

550 

700 

850 

1000 


2000 

2500 

5000 

10K 

20K 

5 OK 



# Frames 
with #C - N 

22 

878 

372 

195 

- 121 

89 

36 

19 

17 

9 

3 

0 

0 

|| 

B 


(3) Channel BSC: p =.045 
Computation 


Total 

H 32 Frames .1000 


Error 

Frames 


8 


Erased 

Frames 


Total Error-Bits: 40 


N 

292 

400 

550 

700 

850 

1000 


2000 

250C 

5000 

10K 


50K 



# Frames 
with #C - N 


992 

763 

534 

415 

327 

205 

140 

110 

56 

25 

15 

5 

fl 

fl 


(4) Channel 
Computation : 


Total 

Frames 


Error 

Frames 


Erased 

Frames 


Total Error Bits: 


N 

HH 

| 

HI 


MB 

| 

■ 

B | 

MM 

AM 

BMf 

|| 

hb 

BBI 

■ 1 

# Frames 
with #C - N 

B 

B 

a 

B 

B 

B 

B 

B 

a 

a 

■ 

■ 

■ 

■ 

■ 


(5) Channel 


H 


Total 

Frames 


Error 

Frames 


Erased 

Frames 


N 

BH 

BH 

HM 


BBI 



BH 

■ ■ 

mm 

IB 

IB 

Hi 

Hfl 

wm 

# Frames 
with #C % N 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 


(6) Channel 
Computation: 


H 


Total 

Frames 


Error 

Frames 


Erased 

Frames 


Total Error Bits: 


N 
















# Frames 
with #C - N 
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Code No. 6 Code Name Maximum Weight Adjoint 

Memory = 35 Rate = % Type Systematic 

Generator Sequences (Octal): 400000000000 

656374423651 


Known Distance Properties: 

d FD= 13 


18 = 


d FREE 


22 


Nature of Construction: Adjoint of code no. 5 


Simulation Results : ^ =20 Total Error Erased 

(1) Channel Gauss: b o ’ H 32 Frames 1000 Frames 0 Frames 7 


Computation: Total Error Bits: 0 

* r 'i r ■» - a ■ w T'"“ g ' g — “ ' — ~a — 


N 

292 

400 

450 

500 

600 

700 

850 

1000 

1200 

150d 

4000 

10K 

25K 



# Frames 
with #C - N 

1000 

971 

902 

818 

662 

541 

421 

335 

267 

207 

65 

17 

ii 

■ 

■ 


Total Error Erased 

(2) Channel BSC: p = .033 H 32 Frames 1000 Frames 0 Frames o 


Computation: Total Error Bits: Q 


N 

292 

400 

550 

700 

850 

1000 

1500 

o 

o 

o 

CM 

2500 

5000 

10K 

20K 

50K 


~~3 

# Frames 
with #C - N 

1000 

880 

385 

193 

122 

86 

34 

21 

17 

8 

2 

0 

0 

■ 

■ 


Total Error Erased 

(3) Channel BSC: p = .045 H 32 Frames 1000 Frames 7 Frames 4 


Computation: Total Error.Bits: 72 

" A — — - ? a 1 a - » 1 — s a nr ' i ' 1 " " * "a 


N 

292 

400 

550 

700 

850 

1000 

1500 

2000 

2500 

5000 

10K 

20K 

5 OK 


zn 

# Frames 
with #C - N 

1000 

991 

757 

534 

427 

354 

210 

151 

114 

53 

28 

12 

n 

■ 

■ 


Total Error Erased 

(4) Channel H Frames Frames Frames 


Computation: Total Error Bits: 

■ ■- 11 " - ■ ~ > n - — "t ---I - g -a— — .... i i . a .,. 


N 

















# Frames 
with #C - N 

|| 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 


Total Error Erased 

(5) Channel H Frames Frames Frames 


Computation: Total Error Bits: 

& - • u- 1 « ■ ■■■■ ■ '■ a » — — g . .y -' T ■ ■ — . .... « — . ■ ■ » t 


N 







Hi 








zn 

# Frames 
with #C - N 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

H 

■ 


Total Error Erased 

(6) Channel H Frames Frames Frames 


Computation: Total Error Bits: 

■ ■ - * ■ — i a r— — y- a — ,: n 8 * ■■■* 5 ^ 1 S 


N 
















# Frames 
with #C - N 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

|| 

■ 

■ 

■ 

■ 
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Code No. 7 Code Name Balanced Code 

Memory = 35 Rate = Jg Type Systematic 

Generator Sequences (Octal): 400000000000 

653125446515 

Known Distance Properties: 

d FD = 13 16 " d FREE - 20 

Nature of Construction: Algorithm A8 


Simulation Results: ^ =20 Total Error Erased 

(1) Channel Gauss: b o H 32 Frames 1000 Frames 3 Frames 5 


Computation: Total Error Bits: 33 

■ — —— — rr- ■ — t i- ■ 1 a 1 1 1 1 a 1 " " 8 1 a ■ * r- r — *-— t - — — » ‘ 


N 

292 

400 

450 

500 

600 

700 

850 

1000 

120C 

1500 

400C 

10K 

25K 



# Frames 
with #C - N 

1000 

971 

908 

814 

665 

537 

410 

399 

263 . 

193 

68 

18 

10 

■ 

■ 


Total Error Erased 

(2) Channel BSC: p = .033 H 32 Frames 1000 Frames 0 Frames 0 


Computation: Total Error Bits: 0 

" ■ * — — - i — — ■ — i — ■ — a « —' i — — 1"" ■■■■ a I"— — a— tr i 


N 

292 

400 

550 

700 

850 

1000 

150d 

2000 

250C 

5000 

10K 

20K 

5 OK 



# Frames 
with #C - N 

1000 

880 

378 

193 

126 

85 

33 

21 

15 

10 

3 

0 

0 

■ 

■ 


Total Error Erased 

(3) Channel BSC: p = .045 H 32 Frames 1000 Frames 12 Frames 2 


Computation: Total Error«Bits: 103 


N 

292 

400 

550 

700 

850 

1000 

1500 

2000 

2500 

5000 

10K 

20K 

50K 


~~3 

# Frames 
with #C - N 

1000 

991 

743 

516 

417 

344 

206 

151 

116 

58 

25 

14 

2 

Hi 

■ 


Total Error Erased 

(4) Channel H Frames Frames Frames. 


Computation: Total Error Bits: 


N 









r~i 






1 

# Frames 
with #C - N 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

HI 

■ 

H 

■ 

■ 

■ 

■ 

■ 


Total Error Erased 

(5) Channel H Frames Frames Frames 


Computation: Total Error Bits: 


N 
















# Frames 
with #C - N 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

|| 

■ 

■ 

■ 

■ 

■ 

■ 

■ 


Total Error Erased 

(6) Channel H Frames Frames Frames 


Computation: Total Error Bits: 


N 

| | 














“H 

# Frames 

with #C - N ; 

■ 

■ 

■ 

■ 

|| 

■ 








■ 

■ 
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Code No . 8 Code Name Balanced Adjoint 

Memory = 35 Rate = Jg Type Systematic 

Generator Sequences (Octal): 400000000000 

732453703632 

Known Distance Properties: 

d FD ' 13 18 1 d FKEE - 22 

Nature of Construction: Adjoint of code no* 7 


Simulation Results: -on Total Error Erased - 

(1) Channel Gauss: b o ~ H 32 Frames 1QQQ Frames Q Frames 3 


Computation: Total Error Bits: Q 


N 

292 

400 

450 

500 

600 

700 

850 

1000 

120C 

1500 

E!i!8I8 

10K 

25K 


zn 

# Frames 
with #C - N 

1000 

973 

900 

800 

649 

536 

412 

324 

■ 

211 

59 

21 

8 

H 

■ 


Total Error Erased 

(2) Channel BSC: p = .033 H 32 Frames 1000 Frames 0 Frames 0 


Computation: Total Error Bits: 0 

-* — '■ I - ' ii • i 11 i g 8 ■ i '■■■■ — 1 t 


N 

292 

400 

550 

700 

850 

1000 

1500 

200C 

2500 

500C 

10K 

20K 

50K 

Ml 


# Frames 
with #C - N 

■ 

873 

376 

191 

114 

87 

37 

20 

15 

7 

2 

0 

0 

1 

■ 


Total Error Erased 

(3) Channel BSC: p = .045 H 32 Frames iqqq Frames 5 Frames 6 


Computation: Total Error .Bits : 64 

' " 1 r " ■ ' ■■ 1 '—* | H ■ '■■■ I 1 mu .. --- — ■'< 1 I" 


N 

292 

400 

550 

700 

850 

1000 

150C 

2000 

250C 

5000 

10K 

20K 

50K 


zn 

# Frames 
with #C - N 

1000 

992 

746 

523 

410 

334 

203 

144 

116 

54 

30 

15 

6 

■ 

■ 


Total Error Erased 

(4) Channel H Frames Frames Frames 


Computation: Total Error Bits: 

, „ ■ ' — — 1 -g % — ■' — nr ■■■■■ » ■■ — a — * r mm 1,1 *■ ‘ i f — 


N 















zn 

# Frames 
with #C - N 

H 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

| 

■ 

■ 


Total Error Erased 

(5) Channel H Frames Frames Frames 


Computation: Total Error Bits: 

« — 1 —-m r ~ r- » — t " 1 4 -a ** — 


N 
















# Frames 
with #C - N 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 


Total Error Erased 

(6) Channel H Frames Frames Frames 


Computation: Total Error Bits: 

—T * ■'T " -*' " - -T- i ... | a . — 8 


N 
















# Frames 
with #C - N 
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Code No. 9 Code Name Balanced Buss gang 2 Code 

Memory = 35 Rate = 3g Type Systematic 

Generator Sequences (Octal): 400000000000 

732443151623 

Known Distance Properties: 

dp D =13 18 = d FRE £ = 20 


Nature of Construction: The code obtained by using Algorithm A8 to extend one 
of Bussgang’s optimal codes. 


Simulation Results v, = 20 Total Error Erased 

(1) Channel Gauss: b' o H 32 Frames 1000 Frames 1 Frames 3 


Computation: Total Error Bits: 7 


N 

292 

400 

450 

500 

600 

700 

850 

1000 

1200 

1500 

4000 

10K 

25K 


zz 

# Frames 
with #C - N 

1000 

973 

899 

799 

648 

521 

395 

326 

262 

199 

57 

19 

10 

■ 

■ 


Total Error Erased 

(2) Channel BSC: p = .033 H 32 Frames 1000 Frames p Frames o 


Computation: Total Error Bits: 0 

■ * 1 ‘ T 1 I I 1 - ... ... ...... - - a > fl- ■ 1"" a — I t 


N 

292 

400 

550 

700 

850 

100C 

1500 

200C 

2500 

5000 

10K 

K8T5 

5 0K 


“1 

# Frames 
with #C - N 

1000 

873 

371 

181 

114 

78 

30 

17 

14 

7 

2 

0 

0 

■ 

■ 


Total Error Erased 

(3) Channel BSC: p = .045 H 32 Frames 1000 Frames 4 Frames 3 


Computation: Total Error„Bits: 38 


N 

292 

400 

550 

700 

850 

1000 

1500 

2000 

2500 

5000 

10K 

20K 

50K 


zz 

# Frames 
with #C - N 

1000 

992 

739 

512 

391 

307 

192 

13c 

105 

43 

25 

13 

3 

■ 

■ 


Total Error Erased 

(4) Channel • H Frames Frames Frames 


Computation: Total Error Bits: 


N 















zz 

# Frames 
with #C - N 

■ 

■ 

■ 

■ 

■ 

9 | 

|| 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 


Total Error Erased 

(5) Channel H Frames Frames Frames 


Computation: Total Error Bits: 

^ - ■ i- t v i 1 a" 1 1 1 — ■ - a i" sr 1 • — i * — n a 


N 















ZZ 

# Frames 
with #C - N 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 


Total Error Erased 

(6) Channel H Frames Frames Frames 


Computation: Total Error Bits: 


N 
















# Frames 
with #C - N 
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Code No. IQ Code Name Balanced Busseang 2 Adjoint 

Memory a 35 Rate = Type Systematic 

Generator Sequences (Octal): 400000000000 

653137244673 

Known Distance Properties: 

d FD = 13 17 = d FREE = 23 

Nature of Construction: Adjoint of code no. 9 


Simulation Results: „ = o 0 Total Error Erased _ 

(1) Channel Gauss: b' o * H 32 Frames 1000 Frames 1 Frames 4 


Computation: Total Error Bits: 19 


N 

292 

-F 

O 

O 

450 

500 

600 

700 

' 850 

1000 


1500 


10K 

25K 



# Frames 
with #C - N 

1000 

971 

906 

814 

662 

550 

408 

334 

256 

185 

69 

22 

14 

■ 

■ 


Total Error Erased 

(2) Channel BSC: p = .0 33 H 32 Frames 1000 Frames 0 Frames 0 


Computation: Total Error Bits: 0 


N 

292 

400 

550 

700 

850 

1000 

O 

O 

2000 

2500 

500C 

10K 

20K 

50K 


~~3 

# Frames 
with #C - N 

1000 

880 

m 

189 

118 

78 

32 

21 

14 

9 

2 

0 

0 

■ 

■ 


Total Error Erased 

(3) Channel BSC: p =,045 H 32 Frames 1000 Frames 3 Frames 7 


Computation: Total Error a Bits: 22 


N 

292 

400 

550 

700 

850 

100C 

1500 

2000 

2500 

5000 

10K 

20K 

50K 



# Frames 
with #C - N 

1000 

991 

740 

506 

402 

317 

187 

138 

117 

46 

24 

12 

7 

■ 

■ 


Total Error Erased 

(4) Channel H Frames Frames Frames 


Computation: , Total Error Bits: 


N 















I 

# Frames 
with #C - N 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

|| 

■ 

■ 

■ 

■ 

■ 

■ 


Total Error Erased 

(5) Channel H Frames Frames Frames 


Computation: Total Error Bits: 


N 
















# Frames 
with #C - N 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 


Total Error Erased 

(6) Channel H Frames Frames Frames 


Computation: Total Error Bits: 


N 
















# Frames 
with #C - N 

■ 

■ 

■ 

■ 

■ 


■ 

■ 

■ 

■ 

■ 

■ 

■ 

pi 

■ 
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Code No. 11 Code Name Non- systematic Code 

Memory = 35 Rate = Type Non-svstematic 

Generator Sequences (Octal): 733533676737 

533533676737 

Known Distance Properties: 

d FD = 11 17 = d FREE 

Nature of Construction: Algorithm A9 


Simulation Results: . =20 Total Error Erased 

( 1 ) Channel Gauss: b / o _ H 32 Frames loop Frames p Frames 5 


Computation: Total Error Bits: p 


N 

292 

400 

450 

500 

600 

700 

850 

1000 

1200 

1500 

4000 

10K 

25K 


_] 

# Frames 
with #C - N 

1000 

968 

909 

835 

676 

567 

445 

358 

292 

225 

70 

17 

9 

■ 

■ 


Total Error Erased 

(2) Channel BSC: p = .033 H 32 Frames 1000 Frames 0 Frames 0 


Computation: Total Error Bits: 0 


N 

292 

400 

550 

700 

850 

1000 

1500 

2000 

2500 

5000 

10K 

20K 

50K 



# Frames 
with #C - N 

1000 

883 

405 

223 

135 

92 

47 

26 

18 

5 

2 

0 

0 

■ 

H 


Total Error Erased 

(3) Channel BSC: p = .045 H 32 Frames loop Frames p Frames a 


Computation: Total Error *Bits: n 

. ... a . B 11 ',. — ■ a ' ■ ' k a 1 ' g —- ■a—' r '■ 1 d 1 — ■ ■■ « • '!■■■ ' 111 * ■ 


N 

292 

O 

O 

550 

700 

850 

I1000 

1500 

2000 

2500 

5000 

1QK 

EBBS 

5QK 



# Frames 
with #C - N . 

1000 

991 

785 

581 

477 

Q 

240 

167 

134 

63 

36 

23 

8 

■ 



Total Error Erased 

(4) Channel BSC: p = .057 H 32 Frames 1QQQ Frames p Frames 249 


Computation: . Total Error Bits: p 


N 

292 

400 

550 

Kilil 

850 

1000 

1500 

2000 

250d 

5000 

10K 

20K 

5 OK 


I 

# Frames 
with #C - N 

1000 


949 

863 

802 

753 

640 

585 

543 

440 

358 

303 

249 

■ 

■ 


Total Error Erased 

(5) Channel H Frames Frames Frames 


Computation: Total Error Bits: 


N 









— 






. 1 

# Frames 
with #C - N 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

|| 

■l 


Total Error Erased 

(6) Channel H Frames Frames Frames 


Computation: ' Total Error Bits: 


N 







— 

zn 

— — 









# Frames 
with #C - N 

|| 

■ 

■ 

II 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 
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Code No. 12 Code Name NASA Code 

Memory = 35 Rate = \ Type Systematic 

Generator Sequences (Octal): 400000000000 

715473701317 


Known Distance Properties: 

d FD = 14 d FREE = 18 


Nature of Construction: The adjoint of the code Forney obtained by using the 
Lin-Lyne algorithm to extend one of Bussgang's optimal codes. 


Simulation Results: . =20 Total Error Erased 

(1) Channel Gauss: b_ o H 32 Frames 1 QQQ Frames p Frames 4 


Computation: Total Error Bits:Q 


N 

292 

400 

450 

500 

600 

700 

850 

1000 

1200 

■«1 

HEBE 

10K 

25K 



§ Frames 
with #C - N 


969 

900 

810 

652 

523 

404 

327 

254 

188 

60 

19 

9 




Total Error Erased 

(2) Channel BSC: p = .033 H 32 Frame s loop Frames 0 Frames 0 


Computation: Total Error Bits: 0 

• c 1 1 1 » I t— 1 f 1 ! 1 — 


N 

292 

400 

550 

700 

850 


1500 

2000 

2500 

5000 

10K 

2 OK 

50K 


Hi 

# Frames 
with #C - N 

1000 

884 

387 

189 

111 

79 

26 

18 

16 

8 

2 

2 

0 

■ 

■ 


Total Error Erased 

(3) Channel BSC: p = .045 H 32 Frames 1000 Frames 2 Frames *+ 


Computation: Total Error. Bits: 12 

— . ' * ■ — — — — *— ■— r rrT* — : — _ 1 t _ .a it— — » r ” t- — a * u — 1 1,1 tt nr— rr 


N 

292 

400 

550 

700 

850 

afiEfil 

1500 

2000 

2500 

500C 

10K 

20K 

5 OK 

mm 

mm 

# Frames 
with #C - N 

1000 

991 

756 

510 

403 

320 

187 

138 

104 

48 

31 

11 

B 

B 

B 


Total Error Erased 

(4) Channel BSC: p = .057 H 32 Frames 1000 Frames 87 Frames 108 


Computation: Total Error Bits: 2071 

“ 1 — i— " 1 1 1 ■» i ■»— — ■ ■■ v - 1 * ■ . — ■ ■ 8 1 & 111 ■ f — — — r I ”“ a 


N 

292 

400 

550 

700 

850 

nasEi 

1500 

KEEK 

2500 

5000 

10K 

20K 

50K 



# Frames 
with #C - N 

1000 

1000 

932 

817 

734 

673 

532 

455 

412 

319 

237 

181 

108 




Total Error Erased 

( 5 ) Channel H Frames Frames Frames 


Computation: Total Error Bits: 

i — — ■ u 1 IT I ” II — i " - I i i T " ' " "i ■ "I 


N 


Ml 

mum 

m 

m 

m 

HI 

mm 

mm 

Hi 

mm 

m 

mm 

mm 

mm 

# Frames 
with #C - N 

fl 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 


Total Error Erased 

(6) Channel H Frames Frames Frames 


Computation: Total Error Bits: 

* — ■ • — - ■ 1 * n r — - a — r - ■■ * " ■ ■' 1 8 — 1 -.. y -. — 


N 
















# Frames 
with #C - N 
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Code No. 13 Code Name Lin-Lvne Code 

Memory = 35 Rate = Type Systematic. 

Generator Sequences (Octal): 400000000000 

653134307713 


Known Distance Properties: 


d 


FD 


14 


19 - d FREE - 22 


Nature of Construction: Forney's extension of the Lin-Lyne algorithm . 


Simulation Results: . =90 Total Error Erased 

(1) Channel Gauss: b' o H 32 Frames 1000 Frames 0 Frames 5 


Computation: Total Error Bits: 0 

—m I * ■ 1 ■ ■ t"""™ —v — — — a ■ -i 


N 

292 

O 

O 

zfr 

450 

500 

600 

700 

850 

FffiF] 

1200 

1500 

n 

10K 

25K 


zn 

# Frames 
with #C - N 

■ 

971 

907 

n 

665 

538 

403 

n 

■ill 

262 

189 

65 

20 

11 

■ 

■ 


Total Error Erased 

(2) Channel BSC: p = .033 H 32 Frames 1000 Frames 0 Frames 0 


Computation: Total Error Bits: 0 


N 

292 

310 

350 

400 

475 

550 

700 

1250 

2500 

5000 

10K 

20K 

50K 



# Frames 
with #C - N 

1000 

1000 

992 

880 

565 

370 

185 

44 

14 

7 

2 

0 

0 

■ 

■ 


Total Error Erased 

(3) Channel BSC: p = .045 H 32 Frames loop Frames 3 Frames 3 


Computation: Total Error,Bits: 31 


N 

292 

310 

350 

400 

475 

550 

700 

1250 

2500 

5000 

10K 

20K 

50K 



# Frames 
with #C - N 

2 | 

1000 

999 

991 

895 

739 

506 

232 

111 

50 

22 

9 

3 

■ 

■ 


Total Error Erased 

(4) Channel BSC: p = .057 H 32 Frames 1QQQ Frames 80 Frames 118 


Computation: , Total Error Bits: 1824 


N 

292 

400 

550 

700 

850 

1000 

1500 

2000 



EES 

20K 

50K 


“H 

# Frames 
with #C - N 

100C 

1000 

927 

809 

738 

666 

542 

477 

432 

330 

272 

206 

118 

■ 

■ 


Total Error Erased 

(5) Channel H Frames Frames Frames 


Computation: Total Error Bits: 

— * — _ - ■ — I" ... , ... . i ""1" ' ■ 1 1 it a — 1 - I — - i 


N 















1 

# Frames 
with #C - N 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 


Total Error Erased 

(6) Channel H Frames Frames Frames 


Computation: ....... Total ^ r ^ or Bits: 


N 
















# Frames 
with #C - N 
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APPENDIX B 


Generalizing from equations (151) , (160) , and (169) , 
(D) and y^ (D) can always be written in the following 


form: 


Y (1) (D) = Q 1 (D) (D n - l) k + R X (D) g(D) (D n - l) k-1 (Bl) 
y (2) (D) = Q 2 (D) (D n -l) k + R 2 (D) h(D) (D n - l) k_1 


for some positive integer k. 
Clearly if k is a power of 2, then 

<D n - l> k = D ta - 1 


(B2) 


and 


n k-1 (k-l)n (k-2)n (k-3)n 

(D - 1) = D + D + D + ... + 1. (B3) 


Therefore 


w [ y^ 1 ^ (D)] ^ k d c 


H 

% 


w„ y (2) (D) > 


k d. 


since 


degree 

V D> . 

< degree 

h(D) ] 

degree 

R 2 (D) 

< degree 

g(D) 


^n _2n 


Let Q X (D) = P 0 (D) + P X (D) D“ + P (D) D + 


3n 

P 3 (D) D + ..., where degree 


P ± (D) 


(B4) 


(B5) 


< n for all i. Then 


for k = 3, 

(1) 


(D) = 


P (D) + (D) D n + P 2 (D)D 2n +... 


3n _2n 
D + D + 


D n + 1 


+ r x (d) g(D) 


D 2n + 1 


(B6) 
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= P q (D) + R X (D) g(D) ] + [ P q (D) + P 1 (D) ] D n + 

P 0 (D) + P 1 (D) + P 2 (D) + R^D) g(D) ] D 2n + [p Q (Dj 

+ P 1 (D) + P 2 (D) + P 3 (D>] D 3n +[p i (D) + P 2 (P)'+ P 3 (D) 

+ P 4 (D) ] D 4n +...+ [p ±-3 (D) + P ± _ 2 (D) + P ± _ 1 (D) 

+ P ± (D) D ln + ... . (B7) 


Let (D) be the coefficient of D 


Then 


C_ (D) + C (D) + C (D) + C (D) + C 0 (D) + C (D) + . . . = R, (D) g(D) 

0 1 4 b o 9 

tB8) 

and 

C 2 (D) + C 3 (D) + C 6 (D) + C 7 (D) +.C 10 (D) + C i;l (D) +... = R 1 (D) g(D) 


Therefore 


C 0 (D) 

+ W R C 1 (D) 

+ «- H 

c 4 (D) 

+ W H 

c 5 (d) + ... £ 

RjlD) 

g(D) 




(BIO) 

C 2 (D >’ 

+ w H [c 3 (d> 

+ W H 

C, (D) 
6 

+ W H 

C (D)l+... £ 

(r>) 

g(D) 




(Bll) 


since a sum of the weights of a set of polynomials is always 
greater than or equal to the weight of the sum of the poly- 
nomials. Hence 


w H y (1) (D) ] > 2d g 


(B12) 


Similarly 
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(B13) 


This same argument can be employed for all values of k. 
The number of coefficient equations similar to (B8) and 
(B9) is always equal to the weight of the polynomial 

(D n -1) ^ \ Therefore if L = w H (D n ~i) k , then 

w y (1) (D) £ Ld 

h ’■ J g 

w H [y (2) (D) > Ldb (B14) 

for all values of k. Note that this agrees with equations 
(B4) when k is a power of 2. 
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